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ABSTRACT 


The  class  of  mathematical  programming  problems  whose  range  is 
In  euclidian  n-space  but  whose  domain  is  an  abstract  space  is 
considered.  A  duality  theory  is  presented  that  relates  the 
constrained  maximization  problem  considered  as  a  function  of 
its  right-hand  side  to  the  associated  Lagranglan  maximization 
problem  considered  as  a  function  of  the  Lagrange  multipliers. 
The  analysis  leads  to  useful  computational  procedures. 

The  constrained  maximization  problem  as  a  function  of  its 
right-hand  side  is 

P(y)  -  Sup  H(z) 
g(z)  l  y 
z  c  S  , 

where  S  is  an  arbitrary  set,  H(z)  a  functional  defined  on 
S  ,  G(z)  a  vector  of  functionals  defined  on  S  ,  and  y  is 
a  vector  in  euclidian  J-space.  The  associated  Lagranglan 
maximization  problem  as  a  function  of  the  Lagrange  multipliers 

L(u)  •  Sup  {H(z)  -  u  •  [G(z)  -  d]} 
zcS 

where  u  is  a  vector  in  euclidian  J-space  and  d  is  the 
specific  value  of  y  for  which  the  solution  of  the 
constrained  maximization  problem  is  sought. 

Various  conditions  for  a  strong  duality,  in  the  sense  that 

Sup  P(y)  ■  Inf  L(u) 
y  <  d  u  >  0 

are  presented.  The  duality  results  and  some  regularity 
properties  of  the  functions  P(y)  and  L(u)  are  used  as  the 
basis  for  computational  procedures.  Dual  and  primal-dual 
algorithms  for  concave  programs  are  given. 

Finally,  some  of  the  theory  presented  is  applied  to  the 
problem  of  optimal  adjustment  of  the  capacity  of  a  firm. 
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CHAPTER  1 


INTRODUCTION 

1,1  RESULTS  TO  BE  PRESENTED 

In  this  paper  we  consider  the  class  of  mathematical  programming 
problems  whose  range  is  in  euclidian  n-space  but  whose  domain  of  definition 
Is  an  abstract  space.  The  motivation  for  including  abstract  spaces  In  the 
analysis  Is  to  obtain  results  for  the  Infinite  dimensional  spaces  of 
economic  interest  as  well  as  the  usual  mathematical  programming  problems 
In  euclidian  n-space.  In  fact,  it  was  the  consideration  of  an  economic 
problem  that  gave  rise  to  some  of  the  theory  to  be  presented.  The  term 
"decomposition  programming"  Is  employed  to  emphasize  the  role  played  in 
the  analysis  by  the  Lagrange  multipliers,  which  historically  have  been 
associated  with  decomposing  large-scale  programs  into  subprograms  of 
more  reasonable  size.  Further  there  is  a  very  close  connection,  which  will 
be  developed  in  the  text,  between  the  computational  procedures  to  be 
presented  and  Dantzlg's  [10]  decomposition  algorithm  for  concave  programs. 

We  will  extensively  investigate  the  relationship  between  the 
constrained  maximization  problem  considered  as  a  function  of  its  right- 
hand  side  and  Its  associated  Lagranglan  maximization  problem  considered 
as  a  function  of  the  Lagrange  multipliers.  Our  analysis  attempts  always 
to  take  that  direction  that  gives  insight  into  new  and  existing  computa¬ 
tional  procedures.  The  constrained  maximization  problem  considered  as 
a  function  of  Its  right-hand  side  is 
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P (y)  •  Sup  H(z) 

(1.1-1)  G(z)  <  y 

z  c  S 

where  S  is  an  arbitrary  set,  H(z)  a  functional  defined  on  S,  G(z) 
a  vector  of  functionals  defined  on  S  ,  and  y  is  a  vector  in  euclidian 
J-space.  When  y  takes  on  a  specific  value,  say  y  ■  d  ,  then  (1.1-1) 
is  merely  the  usual  constrained  maximization  problem.  The  associated 
Lagrangian  maximization  problem  considered  as  a  function  of  its  Lagrange 
multipliers  is 

(1.1-2)  L(u)  -  Sup  (H(z)  -  u  (G(z)  -  d]}  u  >  0 

zcS 

where  u  is  a  vector  in  euclidian  J-space. 

When  y  is  specified  and  S  is  a  set  in  euclidian  n-spate,  the 
conditions  under  which  a  solution  to  (1.1-2)  yields  a  solution  to  (1.1-1) 
are  given  in  the  familiar  Kuhn-Tucker  [19]  saddle  point  theorem.  These 
conditions  have  been  generalized  by  Hurwicz  [17]  to  the  case  were  S  is 
a  set  in  a  linear  space.  However,  establishing  the  optimality  conditions 
and  developing  algorithms  to  satisfy  them  are  two  entirely  different 
problems . 

In  the  development  of  nonlinear  programming  algorithms,  when  S 
is  a  set  in  euclidian  n-space,  an  extremely  important  role  has  been  played 
by  the  duality  theory  of  Wolfe  [29]  and  others.  This  theory  assumes  that 
S  is  an  open  set  and  that  the  functions  are  differentiable;  then  the 
"dual"  problem  becomes 
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Min  ftl(z)  -  u  •  G(z)} 

(1.1-3)  VH(z)  -  u  *  7G(z)  -  0 

u  >  0  z  c  S  . 

However,  as  (1.1-3)  is  not  generally  a  convex  program  even  when  (1.1-1) 
is  appropriately  a  concave  program,  the  role  of  the  "dual"  problem  has 
been  a  passive  one.  That  is,  in  the  course  of  most  nonlinear  programming 
algorithms  "dual"  feasible  points  are  constructed  which  are  used  to 
bound  the  primal  objective  function  and  thus  act  as  a  termination  criteria. 

Alternatively,  when  S  is  a  set  in  euclidian  n-space,  a  much  less 
emphasized  related  problem  has  been  considered  by  Huard  [16],  Falk  [13], 
and  Takahashi  [27].  Falk  refers  to  the  following  as  the  "auxiliary 

* 

problem" 

(1.1-4)  Inf  L(u) 

u>0 

where  L(u)  is  defined  in  (1.1-2).  Huard  [16],  assumes  that  L(u)  is 
differentiable  and  then  shows  that  a  solution  of  (1.1-4)  yields  a  solution 
of  (1.1-1)  directly.  He  also  gives  an  algorithm  for  solving  (1.1-4) 
in  this  case,  Falk  [13],  assumes  that  H(z)  is  strictly  concave,  then 
demonstrates  that  L(u)  must  be  differentiable,  and  hence  that  a 
solution  of  (1.1-4)  yields  a  solution  of  (1.1-1).  Takahashi  [27],  is 
concerned  only  with  equality  constraints  but  obtains  results  similar  to 
those  of  [13].  He  further  considers  linear  programming  problems  and 
develops  a  decomposition  algorithm  that  is  remarkably  similar  to  that  of 
Balas  [3]. 

For  the  case  where  S  is  an  abstract  space,  computational  results 

* 

are  not  very  plentiful.  Everett  [12]  has  developed  the  generalized  Lagrang 
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multiplier  method  to  solve  (1.1-1),  when  S  is  an  arbitrary  set,  using  (1.1-2). 
However,  his  method  is  a  heuristic  one  and  requires  a  complete  search  of  the 
entire  Lagrange  multiplier  space.  This  dilemma  has  been  partially  alleviated 
by  Brocks  and  Geoffrion  [8]  as  they  have  shown  that  Everett's  Lagrange  multi¬ 
pliers  may  be  generated  by  linear  programming;  and  in  fact,  the  technique  is 
essentially  that  of  Dantzig's  [10]  decomposition  algorithm  for  concave  programs. 
However,  Dantzig  [9]  had  already  shown  that  this  technique  could  be  used  for 
programs  defined  on  an  abstract  space  by  applying  it  to  the  linear  optimal 
control  problem. 

In  this  paper  we  present  a  unifying  theoretical  foundation,  and  associated 
computational  procedures,  for  the  concepts  introduced  above.  In  Chapter  2  we 
derive  many  of  the  properties  of  the  functions  P(y)  and  L(u)  .  L(u)  is 
shown  under  rather  weak  assumptions  to  be  a  convex  continuous  function  of  u 
for  all  u  >  0  .  Then  assuming  that  P(y)  is  a  concave  program,  the  behavior 
of  the  resources  utilized  as  a  function  of  their  associated  Lsgrange  multipliers 
is  shown  to  be  regular.  A  conjugate  duality  theory  relating  P(y)  and  L(u) 
is  then  presented.  The  theory  is  related  to  the  work  of  Rockefeller  [23]; 
however,  the  proofs  rely  on  well-known  results  in  the  literature.  Further, 
some  of  the  conditions  obtained  are  not  demonstrable  by  the  usual  separation 
arguments.  The  theory  is  such  that  any  point  that  is  classically  "dual" 
feasible  in  the  sense  of  Wolfe  [29]  is  also  dual  feasible  in  our  sense.  How¬ 
ever,  our  dual  problem  is  always  a  convex  program,  is  defined  without  reference 
to  differentiability,  and  is  meaningful  for  abstract  spaces. 

In  Chapter  3,  we  employ  the  theoretical  results  of  Chapter  2  to  develop 
a  dual  algorithm.  This  algorithm  is  shown  to  be  the  logical  dual  of  Dantzig's 
[10]  decomposition  algorithm  for  concave  programs.  Dantzig's  algorithm  is 
then  extended  to  a  primal-dual  procedure  which  it  is  argued  should  improve 
the  convergence  properties  of  the  algorithm.  Finally,  some  nonconvex  problems 
may  be  handled  by  the  dual  algorithm  and  these  are  pointed  out. 


Since  an  economic  problem  was  the  impetus  for  some  of  the  theoretical 
results  presented,  in  Chapter  4  we  formulate  an  economic  example  for  which 
the  theory  is  applicable.  The  problem,  first  posed  by  Arrow,  Beckmann  and 
Karlin  [1],  is  the  optimal  adjustment  of  the  capacity  of  a  firm.  Ue  formulate 
a  rather  complicated  version  and  then  use  the  notion  cf  a  homothetlc  production 
function,  due  to  Shephard  [24],  to  reduce  it  to  a  more  workable  form.  The 
example  Indicates  that  fairly  complex  problems  may  be  handled  by  the  algorithms 
proposed,  if  efficient  techniques  are  available  for  the  solution  of  the 
Lagranglan  subproblems. 


i  > 


\ 


ns—  ii  . 
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1.2  FORMULATION  OF  THE  CONSTRAINED  MAXIMIZATION  PROBLEM 

We  will  consider  the  following  rather  general  formulation  of  a 
structured  program  defined  on  the  product  of  a  finite  number  of  arbitrary 
sets.  For  applications  these  sets  will  always  be  subsets  of  some 
topological  space.  The  problem  specifically  is 

(1.2-1)  Max  J  Hk(zk) 

kcK 

(1.2-2)  l  Gk(zk)<d  jeJ 

kcK  3  3 

(1.2-3)  *k  c  Sk  ,  k  c  K  . 

The  index  sets  J  ■  {1,2,  ...,  J}  and  K  ■  {1,2,  ...»  K}  are  finite. 

k 

S  ,  k  c  K  are  arbitrary  sets  not  necessarily  in  Euclidian  n-space. 

Hk(zk)  and  Gk(zk),  J  c  J  are  functionals  defined  on  Sk,  and 
d  -  (d^,  ....  dj)e  EJ  is  a  given  vector  in  Euclidian  J-space.  Throughout, 
functionals  will  always  be  real-valued;  but  the  Sun  of  a  functional  may  be 
infinite.  Topological  spaces  will  always  be  real  linear  topological  spaces. 
Finally,  if  S  is  said  to  be  a  convex  set,  it  will  be  understood  that  we 
mean  a  convex  subset  of  a  linear  space.  For  convenience,  we  will  often  use 
the  following  notation 

Max  H(z) 

(1.2-4)  G(z)  <  d 

z  c  S 

to  simplify  the  writing  of  (1.2-1),  (1.2-2)  and  (1.2-3).  It  will  be  under¬ 
stood  that  the  two  systems  are  identical. 

A  convenient  interpretation  of  the  above  formulation  is  in  terms  of  the 

k 

so-called  cell  problem.  S  is  then  the  set  of  allowable  strategies  in  the 
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k**1  cell.  Hk(zk)  Is  the  payoff  from  utilizing  strategy  zk  c  Sk  in  the 

kth  cell.  Gk(z)  i®  t^e  amount  of  the  J**1  resource  consumed  by  utilizing 

zk  t  in  the  k^  cell.  (If  Gk(zk)  is  negative,  it  can  be  thought  of 

fch  r  k  k 

as  the  amount  of  the  j  resource  produced.)  £  G.(z  )  <  d.  J  c  J 

kcK  3  3 

requires  that  the  amount  of  resources  utilized  be  bounded  above  by  the 

available  resources.  The  problem  is  then  to  choose  that  set  of  strategies 
1  K  1  K 

(z . z  )  e  S  x  ...  x  S  that  maximizes  the  total  payoff  without 

exceeding  the  available  resources.  The  cell  problem  received  its  name  from 
considering  the  associated  Lagrangian  maximization  problem. 

(1.2-5)  L(u)  -  Sup  j  l  Hk(zk)  -  l  u  [  l  Gk(zk)  -  d  ] 

( kcK  JeJ  3  keK  3  3 

k  k 

*  e  S  ,  k  e  K 

By  interchanging  the  order  of  summation  and  noting  that  the  problem  is 
defined  on  the  product  space,  (1.2-5)  can  be  evaluated  by  solving  subproblems 
over  the  individual  cells  of  the  form 


(1.2-6)  Lk(u)  -  Sup  (Hk(zk)  -l  u,  G, (zk) } 

,k,sk  i‘J  3  1 


k  e  K 


and  noting  that 


(1.2-7) 


L(u)  -  l  L  (u)  +  l  u  d  . 


Note  that  if  S  for  k  e  K  are  convex  polyhedral  sets  in  Euclidian 

k  nk  k  k 

spaces  of  appropriate  dimension,  say  S  c  E  for  k  e  K  ,  and  H  (z  )  , 
k  k 

Gj(z  )  for  j  c  J  and  keK  are  linear  functions,  we  have  the  familiar 
structure  of  a  decomposition  linear  program.  However,  the  optimization 
techniques  to  be  presented  are  basically  for  convex  programming  problems 


as  the  additional  special  structure  present  in  linear  programming  problems 


is  not  utilized. 
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1.  3  THE  OPTIMALITY  CONDITIONS 

We  will  state  without  proof  the  well-known  results  concerning  necessary 
and  sufficient  optimality  conditions  for  functional  programming  on  general 
spaces.  These  conditions  are  the  obvious  generalization  of  the  familiar 
Kuhn-Tucker  conditions  [19].  The  proof  may  be  found  in  Hurwicz  [17]  by 
specializing  his  theorems  V.l  and  V.3.1. 

Theorem  !• 

(i)  Sufficiency 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  , 
G(z)  a  vector  of  functionals  defined  on  S  ,  and  u  and  d 
vectors  in  Euclidian  J-space,  .  If  there  exists 

A  A 

z  c  S  and  u  >  0  such  that 

(1.3-1)  H(z)-u-  [G(z)-d]£.H(z)-u*  [GUJ-dJ^HUJ-u’  [G(z)-d] 

V  u  £  0  VzeS 

then  z  is  optimal  for  the  constrained  maximization  problem 
(1.2-4). 

(ii)  Necessity 

Let  S  be  a  convex  set,  H(z)  a  concave  functional  defined 
on  S  ,  G(z)  a  vector  of  convex  functionals  defined  on  S  , 
and  u  and  d  vectors  in  Euclidian  J-space,  EJ  .  In  order 

A 

that  z  be  optimal  for  the  constrained  maximization  problem 

(1.2-4)  it  is  necessary  that  there  exist  u  >,  0  and  uq  c  E* 

u  >0  such  that 
o  *" 

(1.3-2)  u  H(z)-u' [G(z)-d] >u  H(z)-u* [G(z)-d]>u  H(z)-u* [G(z)-d] 
o  o  o 

Vu>0  VzcS 


Further,  if  there  exists  some  z°  such  that 
(1.3-3)  G(z°)  <  d 

then  uq  >  0  and  (1.3-2)  reduces  to  (1.3-1)  by  dividing  by 
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CHAPTER  2 

THEORETICAL  FOUNDATIONS 

2.1  THE  LAGRANGIAN  MAXIMIZATION  PROBLEM 

The  purpose  of  this  chapter  is  to  develop  some  of  the  important 
relationships  between  the  usual  constrained  maximization  problem  and  the 
associated  Lagrangian  maximization  problem.  The  former  will  be  considered 
as  a  function  of  its  right  hand  side  and  the  latter  as  a  function  of  the 
Lagrange  multipliers.  In  this  section  no  assumptions  concerning  continuity 
or  convexity  are  made.  The  constrained  maximization  problem  (1.2-4)  as  a 
function  of  its  right  hand  side  is 

P(y)  -  Sup  H(z) 

(2.1-1)  G(z)<y 

z  c  S 

where  the  domain  of  definition  of  the  function  P(y)  is 

(2.1-2)  Y  -  {y  |  y  >  G(z),  z  e  S,  p(y)  >  -  «  }  . 

S  la  an  arbitrary  set,  H(z)  a  functional  defined  on  S  ,  and  G(z)  a 
J-dimensional  vector  of  functionals  defined  on  S  .  The  associated 
Lagrangian  maximization  problem,  with  y  ■  0  ,  is 

(2.1-3)  L  (u)  ■  Sup  (H(z)  -  u  •  G(z)}  u  >  0 

0  zeS 

wherfe  u  e  EJ  is  a  vector  in  Euclidian  J-space.  Throughout,  functionals 
will  always  assume  real  values. 

The  following  theorem  is  a  minor  extension  of  the  "Main  Theorem"  of 
Everett  (12  ]  and  relates  the  solution  of  the  Lagrangian  maximization 
problem  to  a  particular  constrained  maximization  problem. 


n 


\ 


Theorem  2: 


Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  ,  and 
G(z)  a  vector  of  functionals  defined  on  S  .  If  z(u)  solves,  the 
Lagrangian  maximization  problem  for  some  u  >_  0  ,  i.e., 


(2.1-4)  H(z(u))  -  u  *  G(z(u))  =  Max/iHz)  -  u  •  G(z)| 

zcS 

then  z(u)  is  optimal  for  the  following  constrained  maximization 
problem 


(2.1-5) 


Max  H(z) 

G(z)  ^  y 
z  e  S 


where 


(2.1-6) 


Yj  ■  G^(z(u))  if  Uj  >  0 
yj  >  Gj (z(u))  if  Uj  =  0 


Proof i 

Since  z(u)  maximizes  the  Lagrangrangian  for  u  £  0 


by  (2.1-4),  we  have 


H(z(u))  -  u  *  [G(z(u))  -  y]  >.  H(z)  -  u  •  [G(z)  -  y]  Vz  e  S  . 

Further,  since  z(u)  is  feasible  for  G(z)  <_  y  by  (2.1-6),  we  have 
for  u  £  0  . 

H(z(u))  -  u  •  [G(z (u) )  -  yj  ^  H(z(u))  -  u  •  (G(z(u))  -  y]  V  u  >  0  . 

Hence  we  have  satisfied  the  sufficient  conditions  of  Theorem  1,  Section 
(1.3),  and  thus  z(u)  solves  the  constrained  maximization  problem. 

Using  Theorem  2,  we  can  now  show  that  solving  the  Lagrangian 
maximization  problem  defines  a  linear  supporting  function  of  the  con¬ 
strained  maximization  problem  considered  as  a  function  of  its  right  hand 
side,  i.e.,  P(y)  . 
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* 


Def Inlt Ions : 

(i)  n_(x)  ■  w  •  x  +  n_ (0)  is  a  linear  supporting  function  of 

x  x 

F(x)  at  some  x  e  T  ,  where  T  is  the  domain  of  definition 
of  F(x)  ,  if 

(2.1-7)  ir_(x)  ■  F(x)  and 

x 


(2.1-8)  ir_(x)  >  F(x)  V  x  e  T 

x 


(ii)  ir_(x)  -  w  •  x  +  ir_(0)  is  a  linear  bounding  function  of 
x  x 

F(x)  ,  where  T  is  the  domain  of  definition  of  F(x)  ,  if 

(2.1-9)  tt_(x)  >  F(x)  V  x  c  r 

x 


Theorem  3: 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  , 
and  G(z)  a  vector  of  functionals  defined  on  S  .  If  z(u)  solves 
the  Lagranglan  maximization  problem  for  some  u  >  0  ,  i.e., 


H(z(u))  -  u  •  G(z(u))  ■  Max(H(z)  -  u  •  G(z))  , 

zcS 


then 


ir_(y)  -  u  •  y  +  Lq(u) 

y 


is  a  linear  supporting  function  of  P(y)  at  y  e  Y  where 
Yj  ■  Gj (z(u) )  if  Uj  >  0 
yJ  ■  Gj^z^u^  if  uj  “ 


0 
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Proof : 


Sup{H(z)  -  u  •  G(z)}  ■  L  (u) 
zeS  ° 


or  equivalently 

H(z)  <  u  •  G(z)  +  Lq(u)  V  z  c  S  . 

Consider  z(u)  defined  above  such  that 

H(z(u))  ■  u  •  G(z(u))  +  L  (u) 

o 

P(y)  -  Sup  H(z)  ■  H(z(u))  =  u  *  y  +  L  (u)  . 
G(z)  <  y  0 

z  e  S 


The  above  holds  with  equality  by  Theorem  2.  Hence,  we  have  that 
p(y)  -  u  •  y  +  Lq(u) 
and  we  now  must  show  that 

P (y)  <  u  •  y  +  Lo(u)  V  y  e  Y  . 

A 

Consider  arbitrary  y  c  Y  .  Since 

H(z)  <  u  •  G(z)  +  Lo(u)  V  z  e  S  , 
we  can  unite 

P(y)  ■  Sup  H(z)  .  <  Sup{u  •  G(z)-+  L  (u)}  <  u  .  y  +  L  (u) 
G(z)  <  y  G(z)  <  y  0 

z  c  S  z  e  S 


where  the  last  inequality  holds  since  u  >  0  .  Noting  that  y  e  Y 
is  arbitrary,  we  have  the  desired  result  that 
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P(y)  <  u  •  y  +  Lo(u)  V  y  e  Y  . 

'.ence  n_(y)  ■  u  •  y  +  L  (u)  is  a  linear  supporting  function 

y  ° 

of  P(y)  at  y  e  Y  . 

If  the  Sup  is  not  attained  in  the  Lagrangian  maximization  problem, 
then  u  •  y  +  L(u)  only  defines  a  linear  bounding  function  of  P(y)  , 
i.e. , 

(2.1-10)  u  •  y  +  L(G)  >  P (y)  V  y  e  Y  . 

We  now  demonstrate  L  (u)  is  a  convex  and  continuous  function  of 

o 

u  for  all  u  >  0  . 

Definition: 

A  function  F(x)  ,  defined  on  a  convex  set  T  ,  is  convex  if  for 
x1,  x2  e  T  and  0  <  a  <  1 

(2.1-11)  F(a  x1  +  (l-a)x2)  <  a  F(xX)  +  (1-a)  F(x2)  . 

Theorem  4: 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  , 

and  G(z)  a  vector  of  functionals  defined  on  S  .  Then  L  (u) 

o 

is  convex  in  u  for  all  u  >  0  . 

Proof : 

1  2 

To  show  L(u)  convex,  let  u  ,  u  e  {u  |u  >  0}  and  0  <  a  <  1 

■  m  m 

L  (a  u*  +  (l-a)u2)  *  Sup{H(z)  -  (a  u^  +  (l-a)u2)  •  G(z)} 

0  zcS 

■  Sup{a(H(z)  -  u1  ■  G(z) ]  +  (l-a)[H(z)  -  u2  •  G(z)]} 
zcS 
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<  Sup  a[H(z)  -  u*  •  G(z)]  +  Sup  (l-a)|H(z)  -  u^  *  G ( z ) ] 
zeS  zeS 

L  (a  u*  +  (l-a)u^)  <  a  L  (u*)  +  (1-a)  L  (u^)  . 
o  ■  o  o 

Hence  L  (u)  is  convex  in  u  for  all  u  >  0  . 
o  ■ 

Definitions : 


(i) 

A  functional 

F(x)  on  a  topological  space  T 

is 

lower 

semi- 

continuous  if 

the  level  set 

(2.1-12) 

{x  |  F(x)  <  a} 

is  closed  for 

each  real  a  . 

(il) 

A  functional 

F (x)  on  a  topological  space  T 

is 

upper 

semi- 

continuous  if 

the  level  set 

(2.1-13) 

{x  |  F (x)  >  a) 

is  closed  for 

each  real  a  . 

Note  that  if  a  functional  is  both  lower  and  upper  semi-continuous 
it  is  continuous. 

Theorem  5: 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  , 

and  G(z)  a  vector  of  functionals  defined  on  S  .  Then  L  (u)  is 

o 

continuous  in  u  for  all  u  >  0  . 


16 


Proof : 

By  the  theorem  due  to  Gale,  Klee,  and  Rockafellar  [IS],  a  convex 

function  is  upper  semi-continuous  on  a  convex  polytope.  Since  the 

set  (u  |  u  >  0}  ,  on  which  Lq(u)  is  defined,  is  a  convex  polytope, 

we  need  only  show  that  L  (u)  is  lower  semi-continuous  on  this  set. 

o 

Assume  that  for  some  real  a  the  level  set, 

(x  |  L  (u)  <  a} 

o  ■ 

is  not  closed.  Then  there  exists  a  sequence  {un}  -►  u  such  that 
un  e  {u  |  u  >  0}  Yn  and 

(*)  L  (un)  <  a  <  L(u)  V  n  . 

o  ■ 

However, 

I*0  (un)  -  Sup{H(z)  -  u"  •  G(z) }  , 
zcS 

or  equivalently 

L  (u11)  >  H(z)  -  un  •  G(z)  V  z  e  S  . 

o  ** 

Since  {un}  -*■  u  ,  for  n  sufficiently  large,  say  n  >  N  ,  we  have 

L  (un)  >  H(z)  -  u  •  G(z)  V  z  e  S,  n  >  N 

o  ■ 

and  hence 

L  <un)  >  Sup{H(z)  -  u  ■  G(z)}  =  L(u)  V  n  >  N 
zcS 

which  is  a  contradiction  of  (*) .  Hence,  L  (u)  is  lower  semi-continuous 

o 

in  u  for  all  u  >  0  ,  which  is  the  desired  result. 

K  * 
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2.2  CONTINUITY  OF  THE  RESOURCE  UTILIZATION  FUNCTIONS 

One  difficulty  in  employing  Lagrangian  optimization  techniques  is  the 
discontinuous  behavior  of  the  resources  utilized  as  a  function  of  their 
associated  Lagrange  multipliers.  However,  in  the  case  of  the  convex  program 
these  discontinuities  can  be  completely  described. 

Definition: 

The  amount  of  the  resource  utilized  as  a  function  of  the  Lagrange 

multipliers  is  G^(z(u))  where  the  vector  G(z(u))  of  such  functions 
is  given  by 

(2.2-1)  H(z(u))-  u  '  G(z(u))  -  Msx/h(z)  -  u  •  G(z)l. 

zeS  *  > 

We  now  introduce  convexity  assumptions  not  previously  made  and  prove  the 
well-known  result  that  a  concave  program  is  a  concave  function  of  its  right 
hand  side. 

Definition: 

A  functional  F(x)  defined  on  a  convex  set  T  is  (strictly)  concave 
1  2 

if  for  x  ,x  c  T  and  0  <  a  <  1 

(2.2-2)  F(a  x*  +  (1-a)  x^)  ^aF(x^)  +  (1-a)  F(x^)  . 

Theorem  6: 

Let  S  be  a  convex  set  and  G(z)  a  vector  of  convex  functionals 
defined  on  S  .  If  H(z)  is  a  (strictly)  concave  functional  defined 
on  S  ,  then  P(y)  is  a  (strictly)  concave  function  defined  on 
Y  -  (y  |  y  £  G(z) ,  z  e  S,  p  (y)  >  -  »  }  . 

Proof : 

Let  y\  y^  c  Y  and  define  ya  *  a  y^-  +  (l-a)y^  , 
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12  Q 

where  0  <  a  <  1  .  Let  z  ,  z  and  z  be  optimal  solutions  to 

1  2  o  1 

(2.1-1)  corresponding  to  y  ,  y  ani  y  respectively.  Since  zA 

2 

and  z  are  feasible  and  G(z)  convex,  then  for  0  <  a  <  1  we  have 

G(a  4  (l-a)z2)  <  a  G(z^)  +  (1-a)  G(z2)  <  a  y*  4-  (l-a)y2  . 

12  12 
Hence  a  z  4-  (l-a)z  is  feasible  for  P(a  y  4-  (l-a)y  )  ,  but  not 

necessarily  optimal.  Thus 

H(z°)  >  H(a  z1  4-  (l-a)z2)  . 

If  H(z)  (strictly)  concave  then 

H(za)  >  H(a  z1  4-  (l-o)z2)  >  a  H(z*)  4-  (1-a)  H(z2) 

(>) 

Thus  P(a  y1  4-  (l-a)y2)  >  a  P(y1)  4-  (1-a)  P(y2)  . 

(>) 

Using  Theorems  3,  5,  and  6,  we  now  demonstrate  that  under  the  usual 
convexity  assumptions,  if  one  or  more  of  the  resource  utilization  functions 
are  ^ .scontlnuous  at  some  u  ,  then  we  are  in  fact  on  a  linear  segment  of  the 
function  P(y)  .  Further,  if  the  objective  function  of  the  constrained  max¬ 
imization  problem  is  strictly  convex,  then  the  resource  utilization  functions 
are  continuous.  Hence,  the  following  two  theorems. 

Theorem  7 : 

Let  S  be  a  convex  set,  H(z)  a  concave  functional  defined  on  S  , 

G(z)  a  vector  of  convex  functionals  defined  on  S  ,  and  let  the  Indicated 
Max  exist  for  u  c  {u  |  u  >  0}  0  N(u)  ,  where  N(u)  is  a  neighborhood 
of  u  .  If  some  G^(z(u))  is  discontinuous  at  u  ,  where  the  vector 
C(z(u))  of  such  functions  is  defined  by 
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H(z (u) )  -  u  •  G (z (u) )  -  Max{ll(z)  -  u  ■  G(z)}  , 

zeS 

that  is i  there  exists  a  sequence  { un }  -*■  u  as  n  00  such  that 

Lim  inf  G(z(un))  ■  y  4  y  “  Lim  sup  G(z(un)) 
n  “  n  -►  « 

then  for  0  <  «  <  1  , 

P(a  y  +  (l-a)y)  *=  a  P(y)  +  (l-°)  P(y). 

Proof : 

*  n^  "  u°  *  y  +  V”"> 

y 

is  a  linear  supporting  function  of  P(y)  at  yn  *  G(z(un))  by  Theorem 
3.  As  {u11}  -*  u  ,  LQ(un)  -*•  Lq(u)  by  Theorem  5,  thus 

Tf.(y)  -  *_(y)  *  u  •  y  +  Lq(u)  . 

y  y 

Hence,  u  •  y  +  L^Cu)  is  a  linear  supporting  function  of  P(y)  at 
both  y  and  y  ,  but  y  4  y  by  the  assumed  discontinuity  of  some 
G^(z(u))  at  u  .  For  0  <  a  <  1  , 

a:  u  ’  y  +  Lq(u)  =  P(y) 

A  A 

(1-a):  u  •  y  +  Lq(u)  «=  P(y) 

A  A 

Thus,  u(a  y  +  (l-a)y)  +  Lq(u)  *  a  P(y)  +  (1-a)  P(v)  and  by  the 
concavity  of  P(y)  ,  implied  by  Theorem  6  and  the  concavity  of  H(z)  , 

we  have 

(*)  u  (a  y  +  (l-a)y)  +  Lq(u)  <:  P(a  y  +  (l-a)y)  . 
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Now  since  u  *  y  +  L  (u)  is  a  linear  supporting  function  of  P(v) 

o 

u  •  y  +  Lq(u)  >_  P(y)  V  y  c  Y  . 

A 

(a  y  +  (l-a)y)  e  Y  ,  since  G(z)  is  a  vector  of  convex  functions  and 
S  is  a  convex  set.  Hence 

A  A 

u  (a  y  +  (l-a)y)  +  Lq(u)  £  P(a  y  +  (l-a)y) 
which,  with  (*),  implies 

A  A 

u  •  (a  y  +  (l-a)y)  +  Lq(u)  «  P(a  y  +  (l-a)y) 

A  A 

a[u  *  y  +  Lo (u)  ]  +  (l-a)[u  *  y  +  Lo(u)]  ■  P(a  y  +  (l-a)y) 
a  P(y)  +  (1-a)  P(y)  «  P(a  y  +  (l-a)y)  . 

Theorem  8: 

Let  S  be  a  convex  set,  G(z)  a  vector  of  convex  functionals  defined 
on  S  ,  and  let  the  indicated  Max  exist  for  u  t  (u  |  u  >  0)  fl  N(u)  , 
where  N(u)  is  a  neighborhood  of  u  .  If  H(z)  is  a  strictly  concave 
functional  defined  on  S  ,  then  G(z(u))  ,  defined  by 

H(z(u))  -  u  *  G(z(u))  ■  Max  {H(z)  -  u  '  G(z)}  , 

zeS 

is  a  vector  of  functionals  continuous  at  u  . 

Proof: 

Since  H(z)  is  strictly  concave,  H(z)  is  also  concave  and  the 
hypothesis  of  Theorem  6  is  satisfied.  Therefore,  if  some  Gj(z(u)) 
is  discontinuous  at  u  ,  that  is,  there  exists  a  sequence  {un}  -*  u 


such  that 
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11m  inf  G(z(un) )  -  y  <f  y  -  Um  sup  G(z(u)) 
n  -*•  “  n  -  - 

then  for  0  <a  <  1  , 

P(a  y  +  (l-a)y)  -  a  P(y)  +  (l-a)  P(y)  . 

But  H(z)  Is  strictly  concave  Implies  P(y)  is  strictly 
Theorem  6  and  thus  for  y  4  y 

P(a  y  +  (l-a))  >  a  P(y)  +  (l-a)  p(y) 

which  Is  a  contradiction.  Hence  G(i(u))  is  a  vector  of 
continuous  at  u  . 


concave  by 


functions 
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2.3  CONJUGATE  DUALITY  THEOREMS 

In  this  section  we  join  the  present  trend  toward  a  general  symmetric 
duality  theory  and  present  three  computationally  useful  duality  theorems, 
the  proofs  of  which  rely  on  well-known  results  in  the  literature.  In  order 
to  avoid  complicating  the  presentation  with  existence  considerations,  we  will 
for  the  present  employ  Sup  and  Inf  instead  of  Max  and  Min.  The  constrained 
maximization  problem  will  be  called  the  primal  problem. 

(2.3-1)  Sup  H(z) 

G(z)  <  d 
z  c  S 

The  feasible  region  of  the  primal  problem  is  the  set 
(2.3-2)  Z  =  {z  |  z  e  S  ,  G(z)  <  d  ,  H(z)  >  -  •}  . 

The  associated  Lagrangian  maximization  problem  when  the  right  hand  side  of 
the  constrained  maximization  problem  is  explicitly  included  is 

(2.3-3)  L(u)  -  Sup  (H(z)  -  u  .  (G(z)  -  d])  . 

zcS 

We  now  introduce  a  related  problem  first  treated  by  Nuard  [16]  and  referred 
to  by  Falk  [13]  as  the  auxiliary  problem.  It  will  here  be  called  the  dual 
problem. 

(2. 3-4)  Inf  L(u) 

u  >  0 

ms 

The  feasible  region  of  the  dual  problem  is  the  set 
(2.3-5)  U  ■  {u  |  u  >  0  ,  L(u)  <  +  °°}  . 

To  show  that  (2.3-1)  and  (2.3-4)  can  indeed  be  considered  as  dual  to  one 
another,  we  demonstrate  the  usual  results  concerning  weak  ordering  of  the 
objectives,  existence,  infeasibilitv,  and  optimality.  The  following  theorem 
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is  essentially  due  to  Karamardian  [ 18] ;  however,  here  the  domain  of  definition 
of  the  functionals  is  not  restricted  to  Euclidian  n-space. 

For  completeness  we  adopt  the  following  conventions,  which  may  in  fact 
be  argued  by  contradiction. 


(2.3-6)  Sup  H(z)  ■  -  <*> 

z  c  d> 

(2. 3-7)  Inf  L(u)  =  +  00 

uc4 

Theorem  9: 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  ,  G(z) 
a  vector  of  functionals  defined  on  S  ,  and  let  Z  and  U  be  defined 
by  (2.3-2)  and  (2.3-5)  respectively.  Then 

(i)  Weak  Duality 


Sup  H(z)  £  Inf  L(u) 
zcZ  UE.U 


(ii)  Existence 

If  Z  |l  ^  and  U  t  $  then  both  primal  and  dual  have  finite 
optimal  solutions,  i.e., 


-  *  <  Sup  H(z)  <  +  ®  and  +  00  >  Inf  L(u)  >  -  « 
zcZ  ueU 


(iii)  Infeaeibility 


(a) 

If 

u  4 

4 

and 

Inf  L(u)  *=  -  ®  , 
ueU 

then 

Z  -  4 

(b) 

If 

z 

4 

and 

Sup  H(z)  ■=  +  °»  , 
zcZ 

then 

U  =  4> 

(iv)  Optimality 

A  A  A  A  A 

If  z  e  Z  ,  u  c  U  ,  and  H(z)  «=  L(u)  ,  then  H(z)  =  Sup  H(z) 

zf.Z 


and  L(u)  E  Inf  L(u) 
ueU 


Proof : 


(i)  Consider  any  z  c  7.  and  u  e  U 


Since  the  Sup  is  not  necessarily  attained  at  z  ,  we  have 

H(z)  -  u  •  (G(z)  -  d]  £  L(u)  . 

z  c  Z  Implies  G(z)  <*  d  and  u  e  U  implies  u  £  0  . 

Therefore  -  u  •  (G(z)  -  d]  £  0  and  hence 

H(i)  <  H(z)  -  u  •  [G(0  -  d]  <  L(5) 

Since  z  c  Z  and  u  e  U  are  arbitrary  we  have 

Sup  H(z)  <_  Inf  L(u) 
zcZ  ucU 

(ii)  For  any  z  e  Z  and  u  e  U  ,  we  have  from  (i)  and  the  definitions 

of  Z  and  U  that  -  «  <  Sup  H(z)  _<  Inf  L(u)  <_  L(u)  <  +  • 

zcS  ueU 

+  *  >  Inf  L(u)  _>  Sup  H(z)  >  H(z)  >  -  ® 
ucU  zcS 

(iii)  Follows  immediately  from  (ii)  by  contradiction 

A  A 

(iv)  Using  (i)  and  the  fact  that  z  and  u  may  not  be  optimal  for  the 
primal  and  dual  respectively,  we  have 

H(z)  <Sup  H(z)  <  InfL(u)  <  L(u) 
zcZ  ueU 

A  A 

But  since  H(z)  ■  L(u)  ,  equality  must  hold  throughout  which  is  the 


desired  result. 
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In  order  to  obtain  the  strong  duality  result  that  the  objective 
function  of  the  primal  is,  under  the  appropriate  assumptions,  equal  to  the 
objective  function  of  the  dual,  we  will  employ  some  of  the  general  ininimax 
theorems  due  to  Sion  [26]  and  Fan  [ 14 ] .  However,  first  we  must  prove  a 
minor  extension  of  a  lemma  due  to  Karamardian  [  18] .  Here  again  the  domain 
of  definition  of  the  functionals  is  not  restricted  to  euclidian  n-space. 


Lemma  10: 

Let  S  be  an  arbitrary  set,  H(z)  a  functional  defined  on  S  ,  and 
G(z)  a  vector  of  functionals  defined  on  S  .  If  and  K_  are 

defined  by 

K  -  ju,  z  |  z  e  S,  H(z)  -  u  •  (G(z)  -  d]  -  Inf  {H(z)  -  v  •  (G(z)  -  d]  > 

1  v>0 

c 

Kj  ■  (u,  z  |  z  e  S,  u  >  0,  G(z)  <  d,  u  •  [G(z)  -  d]  =  0}  , 
and  the  set  {z  |  z  e  S,  G(z)  <  d}  is  not  empty,  then  . 

Proof : 

(i)  Show  Kx  c  K2 

Let  (z,  u)  e  K,  be  arbitrary.  Hence  z  e  S,  u  >  0  and 

X 

H(z)  -  u  •  [G(z)  -  d]  =  Inf  {H(z)  -  v  •  [G(z)  -  d ] } 

v>0 


which  implies 

(*)  0  <  (u  -  v)  •  (G(z)  -  d]  V  v  >  0  . 


Hence  Gj(z)  <  dj  V  j  e  J  ,  since  if  not,  setting 


Vj  *  UJ  w^en  Gj(z)  <  dj  and 
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9 


when  G^(z)  >  ^  * 

leads  to  a  contradiction  of  (*).  Finally,  u  >  0  and  G(z)  <  d 

■ 

imply  u  •  (G(z)  -  d]  £  0  ,  and  letting  v  ■  0  in  (*)  we  have 
u  •  (G(z)  -  d]  >_  0  ;  hence  u  •  (G(z)  -  d]  ■  0  . 

Therefore  (z,  u)  e  and  C  ^2  * 

(ii)  Show  K2  <=  ki 

Let  (z,  u)  e  be  arbitrary.  Hence  z  c  S  and 
H(z)  -  u  •  [G(z)-d)'»H(z)i<H(z)  -v  •  [G(z)  -  d]  Vv  >  o  • 

Or  equivalently, 

H(z)  -  u  *  (G(z)  -  d]  «  Inf  H(z)  -  v  ’  [G(z)  -  d]  ,  since  u 

v  >o 

m 

Therefore  (z,  u)  e  and  K2  C  K1  * 

(i)  and  (ii)  imply  . 

We  may  now  prove  the  following  duality  theorems.  Euclidian  J-space 
has  the  usual  topology  throughout,  and  S  has  an  unspecified  topology  in 
Theorems  11  and  12,  while  it  is  merely  a  convex  set  in  Corollary  13. 

Definitions : 

(i)  A  functional  F(x),  defined  on  a  convex  set  T  ,  is  quasi-convex  on  P 

1  2 

if  for  x  ,  x  c  T  and  0  £  a  1 

(2.3-8)  F(axJ  +  (1  -  a)x2)<  Max  (F(x1),  F(x2)] 

(ii)  A  functional  F(x),  defined  on  a  convex  set  T  ,  is  quasi-concave  on 

1  2 

T  if  for  x  ,  x  e  T  and  0  ^  a  <  1 

(2.3-9)  F(axX  +  (1  -  a)x2)  >  Min  (FCx1),  F(x2)]  . 


I  V 
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Theorem  11:  Duality  Theorem  (1) 

Let  S  be  a  convex,  compact  subset  of  a  topological  space  and 
H(z)  -  u  *  [G(z)  -  d]  a  functional  defined  on  S  x  (u  c  |  u  >  0)  , 
quasi-concave  and  upper  semi-continuous  in  z  for  all  u  ^  0  .  Then 

Sup  H(z)  ■  Inf  L(u) 

G(z)  <,  d  u>0 
z  e  S 

Proof: 

Sin.  e  H(z)  -  u  •  [G(z)  -  d]  is  linear  and  continuous  in  u 
for  all  z  c  S  ,  it  is  quasi-convex  and  lower  semi-continuous 
Jn  u  for  all  z  t  S  .  Noting  that  H(z)  -  u  *  [G(z)  -  d]  is 
quasi-concave  and  upper  semi-continuous  on  S  for  all  u  ^  0 
and  that  S  is  compact,  we  have  satisfied  the  conditions  of 
Sica's^  tniniraax  theorem  [26],  and  we  have 

(*)  Sup  Inf{H(z)  -  u  •  [G(z)  -  d]}  *=  Inf  Sup{H(z)  -  u  ’  [G(z)  —  d  ] }  . 
zeS  u^O  u>0  zeS 

The  right-hand  side  is  by  definition  the  dual  problem;  and  applying 
Lemma  10,  the  left-hand  side  reduces  to  the  primal  problem.  Hence 
when  the  primal  problem  is  feasible,  we  have 

Sup  H(z)  *  Inf  L(u) 

G(z)  <  d  u>0 
z  c  S 

If  the  primal  problem  is  infeasible  due  to  S  •=  $  ,  then  the  desired 
result  is  immediate  from  (2.3-6).  Otherwise, 

^See  Lemma  A  of  Appendix  A  for  a  statement  of  Sion's  theorem. 
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if  there  does  not  exist  a  primal  feasible  solution,  then 


Inf  { H (z)  -  u  •  (G(z) ] } 
u>0 


Y  z  e  S 


which  implies  that  Inf  L(u)  -  -  «  by  (*) ;  and 

u>0 

noting  equation  (2.3-6),  the  proof  is  complete. 

We  define  the  graph  of  the  function  P(y)  as  the  set 


(2.3-10) 
recalling  that 
(2.3-11) 


Y°  ■  <yQ.y  I  y„  <  p(y) ) 


p(y)  -  Sup  H(z) 
G(z)  <  d 
z  c  S 


and 


(2.3-12)  I  ■  (y  |  y  >  G(z),  z  E  S,  P(y)  >  -  «}  . 

Thggtgm  1?:.  Duality  Theorem  (ii) 

Let  S  be  .  convex  space,  HU)  a  concave  functional  defined  on  S  ,  and 
G(a)  a  vector  of  convex  functionals  defined  on  S  .  If  y°  ,  the  graph 
of  P(y),  is  closed  then 


Sup  H(z)  -  Inf  L(u) 
G(z)  ^  d  u  ^  0 
z  c  S  “ 


Proof: 


Since  the  graph  of  P(y)  is  closed  and  convex,  it  has  a  support  at 
every  boundary  point.  Hence  if  the  primal  problem  has  a  feasible  solution 
it  has  an  optimal  solution.  By  the  necessary  conditions  of  Theorem  1, 
Section  (1.3),  if  z  E  S  is  optimal  for  the  primal  problem  then  there 
exists  u  >  0  and  u  e  E  ,  u^  >  0  such  that 
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*  A  A  A  ■* 

u  II  (z)  -  u  [G(z)  -  <l]  r>  u  II  (z)  -  u  •  (G(z)  -  d]  _> 

V°u  >  0 

t-r 

A 

^  u  H(z)  -  u  •  [G(z)  -  d]  , 

‘=l  o  \r  r- 


Equivalently , 

Inf  {u  H(z)  -  u  •  (G(z)  -  1]}  =  Sup  {u  H(z)  -  u  •  (G(z)  -  d ] } 
u>0  0  zcS  0 


which,  noting  that  z  and  u  are  particular  values,  implies 


Sup  Inf  {u  H(z)  -  u  *  [G(z)  -  d]}  >_ 
zcS  u>0  ° 

ts 

^  Inf  Sup  {u  H(z)  -  u  •  (G(z)  -  d ) }  . 
u>0  zeS  0 

ct 

If  uq  >  0  ,  we  can  divide  by  u^  ,  and  applying  Lemma  10  to  the  left  hand 
side  and  the  definition  of  L(u)  to  the  right  hand  side,  we  have 


Sup  H(z)  >.  Inf  L(u)  . 
G(z)  <  d  u>0 
z  e  S 


Noting  the  weak  duality  of  Theorem  9(1)  equality  must  hold  which  is 
the  desired  result.  If  u^  =  0  ,  let  uq  =  e  ,  and  by  the  same  argument, 

Sup  H(z)  =  Inf  l(— )  r  Inf  L(v)  «=  Inf  I  (v) 

G(z)  <  d  ujtO  ev>0  v>0 

z  e  S 


Hence,  if  the  primal  problen  is  feasible  the  theorem  holds.  If  the 

primal  problem  is  infeasible  due  to  S  =  $  ,  then  the  desired  result  is 

immediate  from  (2.3-6).  Otherwise  we  can  take  u  arbitrarily  large  in 

L(u)  =  Sup  (H(z)  -  u  •  [G(z)  -  d ] } 
zeS 


since  for  any  z 
j  e  J  .  Hence 
complete. 


e  S  an  infeasible  primal  implies  G^(z)  >  d  for  some 

Inf  L(u)  *=-<*>  and  noting  equation  (2.3-6),  the  proof  i^ 
u>0 


In  mathematical  programming  it  is  customary  to  define  Max  and  Min 
over  the  extended  real  line;  that  is,  the  points  and  -®  are 

permissable.  Furthermore,  it  is  always  tacitly  assumed  that  the  primal 
problem  (2.3-1)  may  be  written  with  Sup  replaced  by  Max.  However,  these 
conventions  are  not  sufficient  to  replace  Inf  by  Min  in  the  dual  problem  as 
Slater's  [17]  famous  counter-example  points  out.  To  insure  that  Sup  and 
Inf  may  be  replaced  by  Max  and  Min  respectively  in  the  above  duality  theorems, 
we  must  assume  a  constraint  qualification.  Noting  the  previous  proof  and 
the  necessary  conditions  of  Theorem  1,  we  get  the  following  expected  result. 

Corollary  13;  Duality  Theorem  (iii) 

Let  S  be  a  convex  set,  H(z)  a  concave  functional  defined  on  S  , 
and  G(z)  a  vector  of  convex  functionals  defined  on  S  .  If  there 
exists  an  optimal  solution  to  the  primal  problem  and  z°  such  that 
G(z°)  <  d  ,  then 

Max  H(z)  ■  Min  L(u) 

G(z)  <,  d  u  >,  0 

z  €  S 

If  there  exists  an  optimal  solution  to  the  primal  problem  and  a  nonempty 
interior  of  the  constraints,  then  Sup  and  Inf  may  be  replaced  by  Max  and  fin 
respectively  throughout  this  section,  except  in  the  definition  of  L(u)  . 

If  the  Sup  is  always  attained,  which  is  the  case  if  S  is  compact,  then  Sup 
may  be  replaced  by  Max  also  in  the  definition  of  L(u)  .  For  further 
comments  on  the  existence  of  L(u)  ,  see  Section  (2. A). 

Theorem  11  gives  rather  weak  conditions  on  the  functionals  for  a  strong 
duality  theorem,  as  will  be  brought  out  in  Section  (3. A)  on  Nonconvex 
Considerations.  However,  the  condition  of  compactness  of  S  is  not  always 
satisfied.  When  S  is  not  compact  and  the  primal  is  a  concave  program  with 
closed  graph,  Theorem  12  says  that  a  similar  strong  duality  theorem  holds. 


For  completeness  of  the  theory,  we  note  the  following  corollary, 
which  is  an  immediate  consequence  of  any  of  the  duality  theorems  and  of 
equations  (2.3-6)  and  (2.3-7). 

Corollary  14:  Urd>oundedness 

Under  the  assumptions  of  Duality  Theorem  (i) ,  (ii),  or  (iii) 


(i) 

If 

U  =  4> 

and 

z  4 

then 

Sup  H(z)  =  +°° 
zeZ 

(ii) 

If 

Z  =  $ 

and 

u  4 

4> 

then 

Inf  L(u)  =  -°° 
ucU 

We  may  characterize  the  duality  theory  presented  as  a  conjugate  duality 
by  remarking  that  if  the  graph  of  P(y)  is  closed  L(u)  defines  a  linear 
supporting  function  of  P(y)  .  For  emphasis,  the  conclusions  of  the  duality 
theorems  may  be  stated 


Sup  P(y)  =  Inf  l,(u)  . 
y <d  u>0 

Finally,  we  should  point  out  the  relationship  of  our  duality  with  that 
of  Wolfe  [29].  If  S  is  an  open  set  in  Euclidian  n-space,  H(z)  a  concave 
differentiable  function  defined  on  S  ,  G(z)  a  vector  of  convex  differenti¬ 
able  functions  defined  on  S  ,  then  a  necessary  condition  for  a  maximum  or  a 
minimum  of 

(2.3-14)  H(z)  -  u  •  (G(z)  -  d] 

is  that  the  gradient  is  zero,  i.e., 

(2.3-15)  VH(z)  -  u  •  VC(z)  =  0  . 


Recalling  the  uual  problem  of  Wolfe  [39], 
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Min  H(z)  -  u  •  [G (z)  -  d]  , 

(2.3-16)  VH(z)  -  u  •  7G(z)  -  0 

u  £  0  z  c  S 

we  note  that  whenever  L(u)  is  evaluated,  where 

(2.3-17)  L(u)  -  Sup{H(z)  -  u  •  [G(z)  -  d]}  u>0  , 

zcS 

a  feasible  solution  of  (2.3-16)  is  determined.  However,  our  dual  problem 

(2.3-17)  Inf  L(u) 

u>0 

is  always  a  convex  program,  is  defined  without  reference  to  differentiability, 
and  is  meaningful  for  abstract  spaces. 
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2 .  EXISTENCE  AND  GEOMETRICAL  CONSIDERATIONS 

Since  L(u)  =  L  (u)  +  u  .  d  ,  we  know  from  Theorems  4  and  5,  L(u) 
o 

is  convey,  and  continuous  in  u  for  all  u  _>  0  .  This,  coupled  with  the 
duality  results  of  the  previous  section,  suggests  a  computational  procedure 
based  on  solving  the  dual  in  place  of  the  primal  problem.  However,  two 
questions  remain  to  be  considered.  When  does  a  solution  for  the  dual  yield 
a  solution  for  the  primal,  and  how  is  L(u)  evaluated  when  the  Sup  is  not 
attained?  The  following  theorem  partially  answers  the  first  question. 

Theorem  15: 

Let  S  be  a  convex  set,  H(z)  a  strictly  concave  functional  defined 
on  S  ,  and  G(z)  a  vector  of  convex  functionals  defined  on  S  .  If 
u  solves  the  dual  problem  (3.2-4)  and  the  graph  of  P(y)  is  closed, 
then  z(u)  given  by 

H(z(u))  -  u  •  (G(z(u))  -  d]  «■  Max  (H(z)  -  u  •  [G(z)  -  d]} 

zeS 

solves  the  primal  problem  (3.2-1). 

Proof : 

By  Theorem  3,  u  •  y  +  lo(u)  is  a  linear  supporting  function  of  P(y) 
at  y  ■  G(z(u))  .  Hence 

u  •  G(z(u))  +  L  (u)  =  P(G(z(u)))  and 
o 

(*)  u  •  y  +  Lq(u)  ,>  P(y)  Vy  e  Y  «  (y  |  y  >  G(z),  z  t  S,  P(y)  >  -«} 

Further,  since  u  solves  the  dual  problem  and  the  graph  of  P(y)  is 
closed,  we  have  by  Theorem  V-  that 

P(d)  *  Sup  H(z.)  *  Inf  L(u)  c  u  *  d  +  L  (u)  . 

C(z)  <_  d  u  >  0 

z  e  S 


Now,  assuming  that  z(u)  does  not  solve  the  primal  problem,  we  have 
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G(z(u))  y  d  . 

Hence  we  may  write  the  following  string  of  inequalities;  the  first  of 
which  follows  from  the  strict  concavity  of  P(y)  implied  by  the  strict 
concavity  of  H(z)  by  Theorem  6. 

P(a  G(z(u) )  +  (1  -  a)d)  >  a  P(G(z(u)))  +  (1  -  a)  P(d) 

■  u  *  (a  G(z(u))  +  (1  -  a)d]  +  L  (u) 

o 

>,  P(a  G(z(u) )  +  (1  -  a)d) 

The  last  inequality  follows  from  (*) ,  and  exhibits  a  contradiction. 

Hence  z(u)  must  solve  the  primal  problem. 

If  H(z)  is  only  concave  and  u  solves  the  dual  problem,  z(u)  does 
not  necessarily  solve  the  primal  problem.  However,  the  regularity  properties 
demonstrated  in  (2.2)  make  it  possible  to  construct  an  optimal  solution  for 
the  primal.  An  algorithm  for  this  is  given  in  Section  (3.2). 

The  second  question,  concerning  when  the  Sup  is  attained,  is  in  practice 
rather  easily  avoided.  Evaluating  L(u)  is  Itself  an  optimization  problem 
for  which  termination  conditions  to  within  some  preassigned  e  >  0  are 
necessary  for  all  but  extreme  point  techniques  that  terminate  in  a  finite 
number  of  steps.  The  following  theorem  demonstrates  that  if  we  are  within 
e  of  the  true  value  of  L(u)  then  we  are  within  t  of  the  optimal  value 
for  the  associated  constrained  maximization  problem. 

Theorem  16: 

If  z  is  within  e  >  0  of  the  optimal  solution  of  the  Lagrangian 
maximization  problem  for  some  u  >_  0  ,  i.e., 

H(z)  -  u  •  [G(z)  -  d]  +  e  >_  Sun{H(z)  -  u  •  [G(z)  -  d ] }  *=  L(u) 

zcS 
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then  z  is  within  e  >  0  of  the  optimal  solution  of  1'ie  associated 
r  nstrained  maximization  problem,  i.e., 

H(z)  +  c  ^Sup  H ( z ) 

G(z)  <  G  (z) 
z  c  S 

Proof:1- 

H(z)  -  u  •  [G(z)  -  d]  +  e  _>  Sup|ii(z)  -  u  •  (G(z)  -d]\ 

ztS*  f 

Hence  H(z)  -  u  •  G(z)  +  r  >  H(z)  -  u  •  G(z)  V  z  e  S 

or  equivalently 

(*)  H(z)  +  e  >,  H(z)  +  u  •  (G(z)  -  G(z)]  Y  z  t  S 

Since  (*)  holds  for  all  z  t  S  ,  it  must  hold  for  any  subset  of  S  ,  in 

particular 

S  »  {z  |  G(z)  C(z) ,  z  z  S} 

Noting  that  u  >  0  ,  we  have  on  the  set  S 

u  •  (G(z)  -  G(z) 1^0  V  z  e  S 
and  hence  H(z)  +  e  H(z)  VzeS 

or  equivalently 

H(z)  +  e  >  Sup  H(z) 

G(z)  <  G(z) 
z  e  S 

Hence,  even  though  the  Sup  may  not  be  attained  in  the  definition  of 
L(u)  ,  any  optimization  technique  used  to  evaluate  L(u)  that  comes  within 


This  proof  is  due  to  Flverett  [12  }• 
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c  of  the  Sup  is  sufficient  for  computational  purposes. 

Finally,  if  we  are  to  solve  the  convex  dual  problem  in  place  of  the 
primal  problem,  it  will  be  useful  to  know  the  gradient  of  L(u)  at  u 
if  it  exists,  or  at  least  a  subgradient  if  it  does  not. 

Definition: 

A  vector  w  is  a  subgradient  of  a  function  F(x)  at  x  in 

r  ,  where  r  Is  the  domain  of  definition  of  F(x)  ,  if  there 

exists  a  scalar  ir_(0)  such  that 
x 

(3.4-1)  u_(x)  ■»  w  •  x  +  fl_(0) 

X  X 

is  a  linear  supporting  function  of  F(x)  at  x  . 

Theorem  17: 

[d  -  G(z(u))]  is  a  subgradient  of  L(u)  at  u  ,  where 
G(z(u))  is  defined  by 

H(z(u))  -  u  •  (G(z(u))  -  dj  ■  Max{H(z)  -  u  •  [G(z)  -  d]}  ; 

zcS 

and  it  is  assumed  that  the  indicated  Max  exists  for  u  . 

Proof : 

L(u)  *=  Sup  (H(z)  -  u  •  (G(z)  -  d ] }  =  H(z(u))  -  u  •  [G(z(u))  -  d] 
zcS 

£  H(z)  -  u  •  (G(z)  -  d]  V  z  e  S 
Hence,  in  particular, 

L(u)  >,  H(z(u))  -  u  •  (G(z(u))  -  d] 


or  equivalently 
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-L(u)  <_  u  •  [ G (2 (u )  )  -  d)  -  H ( 2 (u) )  V  u 

which  implies  that  there  exists  tt  (0)  such  that 

u 

[G  (2  (u) )  -  d]  •  u  +  ti_  (0)  is  a  linear  supporting  function  of 

u 

-L(u)  at  u  .  Hence  (d  -  G(z(u))]  is  a  subgradient  of 
L(u)  at  u  . 

If  the  indicated  Max  does  not  exist  for  u  ,  then  there  does  not  exist 
a  support  of  P(y)  with  gradient  u  .  However  if  z  comes  within  e  of 
the  optimal  solution,  i.e., 

(3.4-2)  H(z)  -  u  *  [G(z)  -  d]  +  e  ^  Sup{H(z)  -  u  •  [G(z)  -  d]} 

zeS 

then  [d  -  G(z)  ]  can  be  used  as  an  approximate  gradient. 
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CHAPTER  3 

COMPUTATIONAL  PROCEDURES 

3.1  THE  DECOITPOSITION  ALGORITHM  FOR  CONCAVE  PROGRAMS 

The  algorithms  based  on  the  theory  presented  in  the  previous  chapter 
are  very  closely  tied  to  Dantzig's  [10]  decomposition  algorithm  for  concave 
programs,  often  referred  to  as  generalized  programming;  and  a  certain 
familiarity  with  this  work  will  be  assumed.  However,  a  brief  sketch  of  the 
important  concepts  is  in  order  as  the  convergence  of  this  algorithm  will  be 
assumed.  In  our  notation,  the  problem  treated  by  Dantzig  is 

Max  H(z) 

(3.1-1)  G(z)  <  d 

z  c  S 

where  S  is  a  convex  compact  space,  H(z)  a  concave  upper  semi-continuous 
functional  defined  on  S  ,  aud  G(z)  is  a  vector  of  convex  lower 
semi-continuous  functionals  defined  on  S  .  Actually,  Dantzig  restricts  S 
to  be  a  subset  of  Euclidian  n-space  and  the  functionals  to  be  continuous. 
However,  the  first  restriction  is  unnecessary  for  his  proof,  as  he  himself 
points  out  in  [9];  and  the  second  may  be  weakened  since  a  lower  semi-continuous 
function  is  bounded  below  on  a  compact  set. 

The  technique  used  to  solve  (3.1-1)  is  a  generalization  of  the  simplex 
method  for  linear  programming  in  which  the  coefficients  for  a  column  may  be 
merely  points  drawn  from  a  convex  set.  Consider  the  following  generalized 
linear  program  in  the  variables  Xq  and  s  ,  which  is  equivalent  to  (3.1-1). 

Max  y  X 
Jo  o 

y  X  +  Is  «  d 
1  o 

X  •=  1 

o 

X  >  0,  s  >  0 


(3.1-2) 
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where  (yQ ,  y)  is  a  J  +  1  dimensional  vector  drawn  from  the  convex  set 
(3.1-3)  Y°  =  {yo,  y)  |  yQ  <,  H(z),  y  >,  G(z),  z  e  S)  .+ 

To  initiate  the  computation,  we  assume  we  have  a  nondegenerate  basic  feasible 
solution  to  (3.1-2),  i.e., 

(3.1-4)  3  z  c  S  such  that  G(z  )  <  d  . 

o  o 

Using  the  Simplex  multipliers  u  associated  with  this  basis,  the  usual 
pricing  out  mechanism  becomes  the  following  optimization  problem, 

(3.1-5)  L(u)  =  Max  (H(z)  -  u  •  [G(z)  -  d]}  «=  H(z)  -  u  •  (G(z)  -  d]  . 
zcS 

Hence  a  new  point  (G(z),  G(z)]  c  Y°  is  generated  and  a  new  basis  and  asso¬ 
ciated  simplex  multipliers,  optimal  with  respect  to  the  points  generated  thus 
far,  may  be  calculated;  and  the  procedure  continues.  The  algorithm  may  be 
stated  as  follows. 


Algorithm  1:  Decomposition  Algorithm  for  Concave  Programs 

Step  (0)  Let  R^  ■  {0}  be  an  index  set  whose  only  element  is 

associated  with  z  defined  in  (3.1-4).  Let  B  be  an  coper 
o 

bound  on  the  objective  function;  and  set  B  -  o>  and  n  c  0  . 
Step  (1)  Solve 

R(u^)  *  Max  y  H(z  )  X 

n  r  r 

rtR(n) 

I£R(n) 


rcR 


l  * 

L  r 


(n) 


X  >  0,  r  e  R,  . 
r  ■=  (n) 


*This  definition  of  Y°  is  equivalent  to  (2.3-10). 
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yielding  an  optimal  solution  X^n^  and  associated  optimal 

simplex  multipliers  u^  ^  .  Define  z^n^  ■  £  z  X^ 

IcR(n)  r  r 

Step  (2)  If  B  -  R(u(n))  <  e  ,  STOP. 

Step  (3)  Solve  the  Lagrangian  maximization  problem 

L  <u(n))  -  Max  J H (z)  -  u(n)  •  [G(z  )  -  d] j 
zeS  '  > 


yielding 
index  set 


z  .  Set  B 
n 


R(n+1)  by 


Min  (L(i/n)),  B) 


and  define  the 


R(n+1) 


U{n}. 


Set  n  ■  n  +  1  ,  and  go  to  Step  (1)  . 

If  a  nondegenerate  basic  feasible  solution  is  not  initially  available, 
a  Phase  I  procedure  that  minimizes  the  degree  of  infeasibility  in  the 
constraints  can  be  used  to  generate  one.  In  this  event,  steps  (1)  and  (2) 
are  at  first  replaced  by  steps  (1')  and  (2'). 

Step  (!')  Solve 


Q(u(n))  -  Max  -  l  n. 

jeJ  3 


ri(n)W,-^dJ  3£J 


l  X 
L  r 


-  1 


reR 


(n) 


>r  i  °.  r  e  R(n)  n  >  0  j 


e  J 


yielding  an  optimal  solution  (X^n\  r/n^)  and  associated 


optimal  simplex  multipliers  u 

,<"> .  i  ; 

rtR(n) 


(n) 


Define 
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Step  (2 ' )  If  Q(u^)  «=  0  ,  end  Phase  I  and  go  to  Stop  (1)  . 

The  theorem  shown  by  Dantzlg  [10]  is 

Theorem  18: 

Let  S  be  a  convex  compact  space,  H(z)  a  concave  upper  semi-continuous 

functional  defined  on  S  ,  G(z)  a  vector  of  convex  lower  semi-continuois 

functionals  defined  on  S  ,  z  an  optimal  solution  to  the  primal 

problem  (3.1-1),  and  let  {u^}  be  the  sequence  of  multipliers 

generated  by  Algorithm  1.  If  there  exists  z  e  S  such  that  G(z  )<  d  , 

o  o 

then 

(3. 1-6)  lim  u(n)  =  u 

n-**> 

(3.1-7)  lim  H(z(n))  =  Max  H(z) 

n-w  G(z)  <  d 

z  e  S 

(3.1-8)  H(z)  -  u  *  [G(z)  -  d]  E  Max{H(z)  -  u  •  [G(z)  -  d]| 

zeS<  » 

The  generalized  programming  technique  is  based  upon  the  fact  that  a 
convex  set  can  be  represented  by  a  convex  combination  of  a  sufficiently 
dense  set  of  the  extreme  points  of  the  set.  Step  (3)  of  the  algorithm 
evaluates  L(u)  which,  by  Theorem  3  of  Section  (2.1)  and  the  compactness 
of  S  defines  a  linear  supporting  function  of 

(3.1-9)  P(y)  -  Sup  H(z) 

G(z)  <,  ’ 
z  c  S 

and  hence  an  extreme  point  of  the  set  Y°  .  The  algorithm  essentially  builds 
up  a  convex  polyhedral  set  within  the  convex  set  Y°.  At  each  iteration,  the 
optimal  solution  over  the  polyhedral  set  is  found  in  order  to  obtain  simplex 
multipliers  which  are  then  used  to  generate  another  extreme  point  of  Y°  , 


2 


and  the  procedure  continues.  Since  the  optimal  value  of  the  primal  objective 
function  is  bounded  above  by  L(u)  ,  being  within  c  of  the  minimum  value 
of  L(u)  generated  thus  far  is  used  as  the  termination  criteria.  The 
algorithm  is  referred  to  as  a  primal  method  in  our  terminology  as  it  works 
entirely  within  the  convex  set  Y°  and  merely  uses  the  dual  as  a  bound. 
Further,  a  primal  method  is  a  two  phase  procedure  that  first  generates  a 
feasible  solution  and  then  an  optimal  solution. 

In  the  statement  of  che  algorithms  in  this  chapter,  if  S  is  compact 
we  will  replace  Sup  by  Max  in  the  primal  problem  and  in  the  definition  of 
L(u)  .  However,  the  constraint  qualification  is  required  to  replace  Inf  by 
Min  in  the  dual. 


\ 
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3.2  A  DUAL  ALGORITHM 

In  the  previous  section,  we  characterized  a  primal  algorithm  as  one  that 
depends  on  the  fact  a  convex  set  may  be  represented  by  a  sufficiently  dense 
set  of  its  extreme  points.  We  now  consider  the  logical  dual  of  these  tech¬ 
niques  and  characterize  a  dual  algorithm  as  one  that  depends  on  the  fact  that 
a  convex  set  may  be  represented  as  the  intersection  of  all  supporting  hyper¬ 
planes.  While  a  primal  algorithm  works  entirely  within  the  convex  set  Y°  , 
where 

(32-1)  Y°  =  {y  ,  y  |  y  <  H(z),  y  >  G(z),  z  t  S)  , 

o  o 

a  dual  algorithm  works  entirely  outside  the  convex  set  Y°  .  Recall  that 
Lhe  dual  problem  is 

(3.2-2)  Inf  L(u)  , 

u  >  0 

where  the  function  L(u)  is  given  by 

(3.2-3)  L(u)  *  Sup  !h(z)  -  u  '  [G(z)  -  d]l. 

zeS  1  I 

Also,  whereas  a  primal  method  is  a  two  phase  procedure,  a  dual  method  moves 
simultaneously  towards  feasibility  and  optimality. 

By  Theorems  3  and  4  of  Section  (2.1)  we  know  that  L(u)  is  a  convex 
continuous  function  of  u  ,  for  all  u  >.  0  .  Further,  in  Section  (2.4),  it 
was  shown  tha*’  whenever  I.(u)  is  evaluated  at  some  u  a  subgradient  of 
L(u)  at  u  is  also  determined.  Thus  we  want  to  minimize  a  convex  continuous 
function,  for  which  at  least  a  subgradient  is  readily  available  at  every 
point,  subject  only  to  nonnegativity  restrictions  on  the  variables.  Tin's 
would  seem  to  imply  that  a  straight-forward  gradient  descent  technique 

I 

would  be  efficient.  Huard  [16]  first  and  then  Falk[ 13}  have  proposed  this 
approach;  however,  in  both  papers  h(u)  is  essentially  assumed  differentiable. 


4't 


The  difficulty  arises  from  the  fact,  pointed  out  in  Section  (2.4),  that  the 

a 

primal  solution  z(u)  associated  with  an  optimal  solution  of  the  dual  may 
not  be  feasible  if  L(u)  is  not  differentiable  at  u  .  Therefore,  any 
proposed  algorithm  that  solves  the  dual  problem  in  order  to  solve  the  primal 
must  also  construct  an  optimal  feasible  primal  solution. 

We  first  give  a  gradient  descent  algorithm  for  solving  the  dual  problem. 
Then  for  those  problems  where  the  solution  z(u)  associated  with  the  optimal 
dual  solution  is  not  primal  feasible,  a  perturbation  algorithm  is  given  that 
constructs  a  primal  feasible  solution.  The  algorithm  for  the  dual  is 

Algorithm  2;  Gradient  Descent  Algorithm 

Step  (0)  Set  u^  ■  +  «  ,  L(u^)  -  +  •  ,  ■  0  ,  and  n  ■  1  . 

Step  (1)  Compute  L(u^)  and  a  .  adient  of  L(u)  at  u^  by 
solving  the  Lagranglan  maximization  problem 

L(u)  ■  Max  }  H(z)  -  u^  •  (G(z)  -  d]| 
ze  S  *  » 

yielding  an  optimal  solution  z(u^)  . 

Step  (2)  If  L(u(n))  <  L(u(n_1))  ,  then  define 

«jn)  *  j  dj  "  CjCitu^))  if  dj  or  Uj(n)>0 

(o  if  GJ(i(u(n)))<dj  and  ujn)-0 

If  -  0  ,  STOP. 

Determine  a  new  set  of  Lagrange  multipliers  by 
u(n+l)  m  u(n)  _  g  g(n)  w^ere  g  jg  a  scalar  such  that 

0  <  6  <_  Min  j  0 

and  0  is  the  maximum  step  size.  Set  n  ■  n  +  1  and  go  to 
Step  1. 
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Step  (3)  If  0  <  L(u(n))  -  L(u(n-1))  <  t  and  |  u(n)  -  u(n_1)|  <  6  , 

where  e  and  5  are  preassigned  numbers,  then  STOP.  Other¬ 
wise,  set  0  -  0/2  and  go  to  Step  (1). 

Theorem  19: 

Let  S  be  a  convex  compact  space,  H(z)  a  concave  functional  defined 
on  S  ,  G(z)  a  vector  of  convex  functions  defined  on  S  ,  and  let 
{u(n)}  be  the  sequence  of  multipliers  generated  by  Algorithm  2. 

If  there  exists  z  t  S  such  that  G(z)  <  d  ,  then 

(3.2-4)  lim  u(n)  ■  u 

n-K» 

(3.2-5)  lim  L(u(n))  -  Inf  L(u) 

n-*«  u>0 

EX 


Proof ; 

L(u^)  Is  a  strictly  decreasing  sequence  of  real  numbers  bounded  from 
below  by  the  primal  objective  function,  which  must  be  finite  since  zq 
is  primal  feasible;  hence  L(u^)  converges.  The  sequence  {u^} 

A  /  \ 

converges  to  u  ,  since  termination  takes  place  only  if  6'  *■  0  or 

|  u(n)  -  u(n  ^  |  <  6  ,  one  of  which  must  occur  since  L(u)  is  a  con¬ 
tinuous  function  of  u  for  all  u  ^  0  by  Theorem  4.  Assume,  for  the 
purpose  of  contradiction,  that  L(u)  is  not  optimal  for  the  dual. 

Then  let 


6  »  dj  -  Gj(z(u))  if  Gj(z(u))  >_  dj  or  Uj  >  0 

A  A 

0  if  Gj(z(u))  <  dj  and  u^  -  0 

A  y  A  A 

where  5^0,  since  we  would  then  be  optimal.  Consider  u  -  9  6  . 
By  the  convexity  of  L(u)  and  the  termination  of  the  algorithm,  we 


have 


and  by  the  definition  of  6  ,  we  have 


Hence  a  subgradient  at  u  is  positive  and  a  subgradient  at  u  -  06 
is  negative.  This  implies  that  there  exists  a  point  in-between  with 

A 

a  subgradient  of  zero.  Hence  either  L(u)  is  optimal  to  the  dual  or 

A  AAA 

there  exists  8  such  that  L(u  -  86)  <  L(u)  and  the  algorithm  would 
not  have  terminated,  which  is  a  contradiction. 

If  u  is  an  optimal  solution  to  the  dual  problem  such  that  6  ■  0  , 
then  by  the  definition  of  6 


(3.2-6) 


*"h 

c 

Lk 

V 

o 

then 

G  (l(u)) 

If  Gj(c(u))  <  dj 

then 

“j 

Therefore,  6-0  Implies  that  z(u)  is  a  feasible  solution  to  the 
primal  problem,  and  hence  optimal  by  Theorem  9(iv)  Section  (2.3).  However, 
if  z(u)  is  not  a  feasible  solution  to  the  primal,  then  essentially  a  Phase  I 
type  procedure  must  be  instituted.  The  following  perturbation  algorithm  will 
construct  the  desired  optimal  feasible  solution  to  the  primal. 
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Algorithm  3: 


Step  (0)  Given  u  ,  the  optimal  dual  solution  resulting  from  Algorithm 

2,  and  z(u)  its  associated  primal  infeasible  solution,  let 

^(0)  S  an  *nc*ex  set  whose  only  element  is  associated 

-  *  (0) 

with  z(u)  .  Set  u  ■  u  and  n  ■  0  . 

Step  (1)  Solve 

Q(u(n))  -  Max  -  l  n. 

JcJ  -1 


l  G  (z  )X  -  n.  <  d  j  c  J 

rcR(n)  ^  r  r  J  “  i 


l  XT  "  1 

rcR,  . 

Xr  >  0  ,  r  t  R(n),  nj  «  0,  J  e  J 
yielding  an  optimal  solution  (X^n\n^)  and  associated 


optimal  simplex  multipliers  .  Define  z''n;  «  £  -  .  (n) 

reR,  *r r 
(n) 

Step  (2)  If  Q(u(n))  -  0  ,  STOP. 

Step  (3)  Otherwise,  Q(u^)  <  0  and  determine  new  Lagrange  multipliers 
by  u(n+1)  ■  u(n)  +  0n^  where  0  is  in  the  interval 


(n) 


u, 


(n) 


0  <  6  <,  Min]  0, 


(n) 


and  0  Is  an  upper  bounded  on  the  step  size. 


Step  (4)  Solve 


L(u^n^)»  Max(H(z)  -  u^  •  [G(z)  -  d ]  } 
zeS 


yielding  an  optimal  solution  z^  .  If 
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0  <,  L(u(n)) 

by  R(n+1)  ’ 
set  0  -  0/2 


L(u)  <  t  ,  then  define  the  index  set  R, 

(n+1) 

R(n)U  and  go  to  Step  (1).  Otherwise, 

and  go  to  Step  (4). 


Theorem  20: 

Let  S  be  a  convex  compact  space,  H(z)  a  concave  upper  semi- 
continuous  functional  defined  on  S  ,  G(z)  a  vector  of  convex  lower 
semi-continuous  functionals  defined  on  S  ,  and  let  {u^}  be  the 
sequence  of  multipliers  generated  by  Algorithm  3.  Then 

(3.2-7)  lim  Q(u(n))  -  0 

n-»«> 


Proof: 

If  at  each  iteration  we  set  u^  -  ,  the  algorithm  reduces  to  a 

Phase  I  generalized  program  and  such  a  procedure  converges  by  Theorem 

A 

18.  However,  by  Theorem  11  of  Section  (2.3)  and  the  fact  that  u 
is  an  optimal  solution  for  the  dual,  we  have 

P(d)  “Sup  H(z)  -  Inf  L(u)  ■  L(u)  , 

(*)  G(z)<d  u  >_  0 

z  c  S 


and  hence  that  the  primal  optimal  must  occur  on  a  hyperplane  whose 

A 

gradient  is  u  .  Algorithm  3  is  merely  a  Phase  I  generalized  program, 

restricted  to  the  intersection  of  the  hyperplane  defined  by  (*)  and 

Y  ,  and  it  converges  by  Theorem  18,  Section  (3.1)  if  an  extreme 
o 

point  of  this  set  is  generated  at  each  iteration.  Steps  (3)  and  (4) 
construct  the  necessary  extreme  point  in  a  finite  number  of  steps  by 
the  continuity  of  L(u).  Hence  the  procedure  converges  and 

lim  Q(u(n))  -  0  . 
n+* 
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3.3  A  PRIMAL-DUAL  ALGORITHM  FOR  CONCAVE  PROGRAMS 

One  of  the  computational  difficulties  with  using  Dantzig's  decomposition 
algorithm  for  concave  programs  is  that  at  times  it  exhibits  poor  convergence 
properties.  Poor  convergence  for  any  particular  problem  is  due  to  the  shape 
of  the  convex  set  Y°  for  that  particular  problem.  Recall 

(3.3-1)  Y°  -  {y  ,  y  |  y  £  H(z),  y  >,  G(z),  z  c  S)  . 

The  duality  theory  presented  in  the  previous  chapter  has  the  geometrical 
property  that  for  a  particular  vector  of  Lagrange  multipliers  u  ,  if  the 
associated  primal  solution  is  not  unique  and  therefore  on  a  linear  segment, 
then  the  associated  dual  solution  is  pointed;  and  vice-versa.  Hence,  when¬ 
ever  one  method  exhibits  poor  convergence  properties,  the  other  should  not 
have  these  difficulties.  Therefore  it  is  believed  that  a  primal-dual  method 
will  overcome  any  of  the  poor  convergence  properties  that  either  nethod  might 
exhibit  independently. 

The  following  algorithm  is  a  combination  of  the  primal  and  dual 
algorithms  presented.  At  each  Iteration,  an  extreme  point  of  the  set  Y° 
is  generated  using  the  optimal  simplex  multipliers  for  that  iteration. 

Then,  Instead  of  returning  to  the  linear  program  as  in  Algorithm  1,  a 
gradient  descent  step  on  L(u)  is  performed.  Hence,  a  second  extreme  point 
of  the  set  Y°  is  generated,  and  finally  the  procedure  returns  to  the  linear 
program  compute  new  simplex  multipliers. 

Algorithm  4  Primal-Dual  Algorithm  for  Concave  Programs 

Step  (0)  Let  *  {0}  be  an  index  set  whose  only  element  is 

associated  with  z  defined  in  (3.1-4).  Set  B  =  ®  and 


n  «  0  . 
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Step  (1)  Set  flag  ■  0  and  solve 

R(u(n))  -  Max  l  H(i  )X 

rcR(n) 

I  JtJ 

rcR(n)  3 

T  X  -1 

“  r 

rcR(n) 

Xr  2  °-  r  E  R(n) 

yielding  an  optimal  solution  ,  and  associated  simplex 

multipliers  u^  .  Define  z^  -  £  z  A 

rER(n) 

If  B  -  R(u(n))  <  c  ,  STOP. 

Solve 

L(u^)  ■  Max  Ih(z)  -  u(n)  '  (G(z)  -  d ] j 
zeS  (  » 

yielding  zfl  .  Set  B  -  Min  {L(u^  ^),  Bj  and  if  B  ■  L(u^), 
set  u  »  u(n)  and  z(u)  ■  zn>  Define  the  index  set  R^n+]_) 

by  R(n+1)  "  R(n)  U  ^ ° 9  "  1  »  then  g0  t0  SteP 

Step  (4)  Define 

Gj (z(u))  if  G^(z(u))  i  dj  or  u^  >  0 
if  Gj(z(u))  <  dj  and  u^  *  0  . 


Step  (2) 
Step  (3) 


Determine  a  new  set  of  Lagrange  multipliers  by 
u(n+D  ■  u  -  06^  where  0  is  a  scalar  such  that 


0  <  0  <,  Min  { 0  , 


,<n) 

J 


S<">  >  0 


and  0  is  the  maximum  step  size.  Set  flag  m  1,  n  *  n  +  1  , 


and  go  to  Step  (3) . 
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If  there  does  not  exist  an  Initial  nondegenerate  basic  feasible  solution, 
as  defined  in  (3.1-4),  then  a  Phase  I  procedure  can  be  used  which  is  also  a 
primal-dual  algorithm.  In  this  event,  stops  (1)  and  (2)  are  at  first 
replaced  by  steps  (1*)  and  (2')  of  Section  (3.1).  Alternatively,  only  dual 
steps  might  be  performed  until  a  feasible  solution  is  generated.  This 
eliminates  the  Phase  I  procedure  and  has  an  intuitive  appeal  since  the  dual 
algorithm  moves  simultaneously  towards  feasibility  and  optimality. 


Theorem  21: 

Let  S  be  a  convex  compact  space,  H(z)  a  concave  upper  scmi-continuous 

functional  defined  on  S  ,  G(z)  a  vector  of  convex  lower  semi-continuous 

* 

functionals  defined  on  S  ,  z  an  optimal  solution  to  the  primal  problem 

(3.1-1),  and  let  {u^}  be  the  sequence  of  multipliers  generated  by 

Algorithm  4.  If  there  exists  z  e  S  such  that  G(z  )  <  d  ,  then 

o  o 

(3.3-1)  lim  u(n)  ■  u 

n-x*> 

(3.3-2)  lim  H(z(n))  -  Max  H(z) 

n-*«  G(z)  <  d 

z  c  S 


(3.3-3) 


Proof : 


H(z)  -  u  ’  (C(z)  -  d]  *  Max(H(z)  -  u  •  [G(z)  -  d]} 

zeS 


Since  the  algorithm  finds  two  extreme  points  of  the  set  Y°  at  each 
iteration  of  the  decomposition  algorithm  for  concave  programs,  it  must 
converge  by  Theorem  18. 
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3.  A  NONCONVEX  CONSIDERATIONS 

In  the  previous  sections  of  this  chapter  the  emphasis  has  been  on 
relating  the  dual  algorithm  presented  here  to  Dantzig's  decomposition 
algorithm  for  concave  programs.  However,  by  Theorems  3  and  A  of  Section  (2.1), 
we  have  that  L(u)  is  a  convex  continuous  function  of  u  for  all  u  £  0  ; 
and  this  result  does  not  explicitly  depend  on  the  nature  of  the  set  S,  cron  the 
properties  of  the  functionals  H(z)  and  G(z)  defined  on  S  .  Hence  the 
dual  problem 

(3.A-1)  Inf  L(u)  , 

u  >,  0 

where  L(u)  is  given  by 

(3.4-2)  L(u)  ■  Sup (H(z)  -  u  •  (G(z)  -  d) > , 

zcS 

may  be  solved  in  place  cf  the  primal  problem  even  when  the  primal  is  not  a 
concave  program.  Since  evaluating  L(u)  defines  a  linear  supporting 
function  of  P(y)  ,  assuming  the  graph  of  P(y)  is  closed.  If  It  can  be 
shown  that  the  solution  of  the  primal  problem  Is  on  the  convex  hull  of 

(3.A-3)  Y°  ■  {yQ,  y  |  yQ  <  H(z),  y  >,  G(z),  z  e  S}  , 

then,  noting  Theorem  12  of  Sectlo::  (2.3),  the  dual  algorithm  may  be  used 
to  produce  this  solution. 

For  that  particular  nonconvex  problem  that  satisfies  the  conditions  of 
Theorem  11,  Section  (2.3),  we  have  the  following  much  stronger  result. 
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Theorem  22: 

Let  S  be  a  convex  compact  space,  H(z)  -  u  •  (G(z)  -  d]  a 
functional  defined  on  S  x  (u  |  u  >  0)  ,  quasl-concavu  and 
upper  semi-continuous  in  z  for  all  u  >  0  ,  and  let 
be  the  sequence  of  multipliers  generated  by  Algorithm  2, 
Section  (3.2).  Then 

(3.4-3)  lim  u(n)  -  u 

n-K» 

(3.4-4)  lim  L(u^)  *  Inf  L(u) 

n-»®  u  >_  0 

Further,  if  there  exists  zq  e  S  such  that  c(;!0)  <  d  » 
then  the  Inf  is  always  attained  and  may  be  replaced  by  Min. 
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CH  ,  rER  4 

DYNAMIC  ECONOMIC  PLANNING 

4.1  THE  OPTIMAL  ADJUSTMENT  OF  THE  CAPACITY  OF  A  FIRM 

As  Indicated  in  the  introduction,  an  economic  problem  gave  rise  to  the 
more  general  results  presented  in  this  paper.  Therefore,  in  this  chapter 
we  formulate  an  economic  example  for  which  the  theory  is  applicable.  The 
particular  problem  was  chosen  because  it  has  been  considered  by  others,  is 
relatively  straight-forward  to  explain,  and  emphasizes  the  dynamio  nature 
of  the  policies  of  a  firm.  It  was  the  desire  to  actually  compute  these 
dynamic  policies  for  the  firm  under  nontrivial  assumptions  that  motivated 
the  theory  presented. 

We  will  consider  a  multi-divisional  firm  where  each  division  produces 

a  separate  commodity.  Further,  we  will  assume  that  the  production  structure 

is  separable  in  the  sense  that  the  policies  of  one  division  only  effect 

the  policies  of  another  through  demands  for  the  same  resources  [7  ].  If 

none  of  the  firm's  resources  are  binding  then  the  policies  of  the  different 

divisions  are  completely  independent.  For  simplicity,  we  assume  that  the 

production  structure  of  each  division  can  be  represented  by  a  homothetic 

production  function  with  an  analytic  form,  and  that  storage  is  not  allowed. 

(See  Appendix  B  for  definitions  and  examples  of  homothetic  production 

functions.)  In  general  the  problem  to  be  considered  is,  given  the  demand 

for  each  commodity  as  a  function  of  time,  determine  the  capacity  of  a  firm 

to  meet  this  demand  as  a  function  of  time  subject  to  budgetary  and  other 

resource  constraints, 
u 

Let  t  (t)  ,  defined  for  t  e  [0,  T)  ,  be  the  demand  at  time  t  for  the 
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k£'n  commodity  of  the  firm;  a  .id  assume  (k(t)  is  continuous  and  has  a  finite 

number  of  relative  maxima  in  the  interval  (0,  T)  .  Let  y  (t)  ,  defined  for 

t  c  (0,  T]  ,  represent  the  capacity  of  the  firm  at  time  t  to  produce  the 
th  k 

k  commodity,  and  let  v  (t)  ,  defined  for  t  c  [0,  T)  ,  be  the  actual 
level  of  production  at  time  t  of  the  commodity.  Since  storage  is  net 
allowed  for  this  simple  model,  we  have 

(4.1-1)  vk(t)  <  Min  (yk(t)  ,  (k(t))  k  c  K  . 

Letting  Fk(4»k(x))  be  the  homothetic  production  function  for  the  k**1  commodity, 

a  nonnegative  vector  x  (t)  of  inputs  at  time  t  can  produce  a  nonnegative 
k  th 

amount  v  (':)  of  the  v  commodity  at  time  t  if  and  only  if 
(4.1-2)  fkf'vk(xk(t)))  >  vk(t)  k  e  K  . 

Assuming  the  supply  of  inputs  is  not  restrictive,  the  net  production  income 
th 

for  the  k  division  at  time  t  (sales  revenue  less  production  costs)  is 
given  by 

(4.1-3)  rk  vk(t)  -  pk(p)  fk(vk't))+  k  c  K 

t  h  k 

where  the  selling  price  of  the  k  '•enir.odity  r  and  the  vector  of  purchase 
prices  of  the  Inputs  p  are  assumed  .instant.  Further,  assume  that  we  can 
increase  capacity  at  a  positive  cost  proportional  to  the  rate  of  increase,  i.e., 

(4.1-4)  ck  yk(t)  if  yk(t)  >  0  , 


See  Appendix  B  for  this  representation  of  a  homothetic  cost  function  and  note 
that  F~ 1  ( • )  A  f(0  . 


1 
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and  decrease  capacity  at  a  salvage  value  proportional  to  the  rote  of 
decrease  but  leas  than  or  equal  to  the  coat  of  an  equivalent  Increase.  1.,.. 


(4.1-5) 


k  k  -k 


Y  c  y  (t)  If  y  (t)  <0  and  0  <  Yk  <  1  . 


Hence  the  present  value  of  the  Income  of  the  fir.  for  all  commodities  over 
the  time  Interval  [0,  T]  is  given  by 


(4.1-6) 


T_ 

I  J  I 

kcK  J.  ' 


rk  vk(t) 


Pk(p)  fk(vk(t)) 


-  c 


At)  fd(yk(t))  +  /(I  .  {(^(t)))]J  e~at  dt 


(4.1-7) 


where  6(z) 


[1  If  z  >  0 

(o  If  z  <  0 
and  a  Is  the  interest  rate. 


The  problem  1.  then  to  choose  the  capacity  function  y(t)  .  with  the  Initial 
capacity  y(0)  end  the  time  horizon  T  known,  that  maximizes  (4.1-6) 
subject  to  (4.1-1)  and  some  typical  constraints.  For  example. 


(4.1-8) 


Lk  <  yk(t)  <  Mk  t  c  [0,TJ  k  e  K 


<4'l's)  kL  J  l°k  *k(t>  M<‘»  +  *k 


(i  - 


<5(yk(t)) 


-at  . 

e  dt  <  B 


Relation  (4.1-8)  bounds  the  rate  at  which 


we  may  change  the  capacity  und 
(4.1-9)  bounds  the  expenditures  on  those  changes  In  capacity  by  a  given  budget. 
If  we  now  let  z(t)  =  y(t)  so  that 


w - 


57 


(4.1-10) 


y (t)  -  y(0)  +  J  r (s)  ds 
0 


we  can  summarize  the  problem  formulated  above  as  follows: 
Example  1 


(4.1-11)  Max 


T 

I  /  | 

kcK  « 


rk  vk(t)  -  Pk(p)  fk(vk(t))  - 


-  ck  zk(t)|fi(zk(t))  +  Yk(l  -  6(2k(t)))]je'otdt 


subject  to 


k 

v, 

k  k 
)*  .v 

Lk  <  zk(t)  <  Mk 

t  c  10,  T)  ) 

0  <  vk(t)  <  Min 

,k(0)  +|zk(s)ds,  tk(t)H 

l 

l  f  |ck  Zk(t)  1 6 (zk(t) )  +  Yk(l  -  6(zk(t))J|  e“at  dt  <  B  . 
kcK  Jf. 


The  above  example  only  has  one  global  constraint,  the  budget  constraint. 
This  convenient  formulation  depends  upon  the  assumption  that  the  available 
Inputs  are  not  restrictive  and  thus  the  minimum  cost  function  at  time  t  for 
the  k1*1  commodity  does  not  depend  explicitly  on  xk(t)  . 

We  will  now  complicate  Example  1  by  allowing  the  input  space  to  be 
bounded.  We  add  constraints  for  those  factors  of  production  that  are  limited 
in  supply.  Let  I  ■  (1,  2,  ...,  1}  be  the  index  set  of  these  factors. 


(4.1-12) 


k  k.  .  .  0  ,  _ 

a^  x^(t)  dt  <  i  e  I 
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A .  2  APPLICABILITY  OF  THE  THF.ORY 

In  order  to  demonstrate  that  the  theory  is  applicable,  we  need  only 
verify  that  the  conditions  of  the  theorems  of  Chapter  3  are  satisfied. 

We  will  assume  that  the  strategies  under  consideration  in  both  examples  are 
chosen  from  the  space  L2  consisting  of  numerical  functions  g  defined 
on  [0,  T]  (to  within  a  set  of  measure  zero)  with  norm  defined  by 


(A. 2-1) 


|  g | |  B  /  |g(t)  |  dt  <  +  «  • 

Jo 


It  is  well  known  that  is  a  complete  normed  linear  space  and  thus  a 

Banach  space. ^  Since  is  also  reflexive,  we  can  use  the  weak*  topologv 

on  L2  ,  under  which  the  strategy  sets  will  be  compact. 

We  will  first  consider  Example  1.  The  strategy  sets  S^tk  e  K  are  convex 
by  the  following  lemma. 

Lemma  23 : 

The  set 


[  z,  v 

L  <  z(t)  <  M 

t  c  [0,  T]  ) 

J 

/  t 

v  \ 

( 

0  <  v(t)  ^  Min  1  y(0)  +  J 

z(s)ds,  5's)j  1 

is  convex  in  (7.,  v)  . 


Proof : 


The  first  constraint  is  linear  in  z  and  the  second  is  linear  in  v 
Therefore,  we  need  only  show  that 


See  Bcrge  [6]  p.  252. 


1 
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Min  |y(0)  z(s)ds,  $(t)j 


Is  a  concave  function  in  z  and  hence 


h + { 


v(t)  -  Min  [y (0)  +  J  z(s)ds,  ?(t) 


is  a  convex  function.  We  have  that  y (0)  + 


^  r(a)d 


s  is  linear  in 


z  and  C(t)  is  constant  in  z  .  Thus  Min  ^y(O)  +  ^  z(s)ds,  £(t)j 

is  a  concave  function  in  z  ,  since  the  minimum  of  two  concave  functions 
is  concave. 

To  establish  that  the  global  constraint  is  a  convex  function  we  prove  the 
following  lemma. 


Lemma  24: 


If  0  <  y  <  1  ,  then  the  capacity  adjustment  cost  function 


J  {c  z(t)[6(z(t))  -  y(l-6 (z (t) ) ] }  e"at  dt 
0 


is  chvu  in 


z  . 


Proof : 

The  result  is  easily  seen  by  graphing  the  Indicated  function  for 
fixed  t  as  follows: 

A  c  z(t)[6(z(t))  -  y(l-  (z(t)))] 

♦  *(t) 

Since  e  at  >  0  V  t  ,  the  integral  of  interest  is  convex  in  z  . 
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The  objective  function  of  Example  1  is  concave  in  (z,  v)  since  the  total 
Income  is  the  sum  of  the  net  production  income 


(A. 2-2) 


|rk  vk(t)  -  Pk(p)  fk(vk(t))J 


I 


which  is  concave  in  v  if  f  (v  )  is  convex  in  v  »  and  the  negative  of 
the  capacity  adjustment  cost,  function 


(4.2-3) 


-  I  f\ck  zk(t)[4(zk<t))  +  Yk(l-S(z\t))]  !e‘ot  dt 
™J0. 


which  is  concave  in  z  by  Lemma  15. 

k 

In  Example  2,  we  have  strategy  sets  k  e  K  that  differ  from  those 
of  Example  1  only  by  the  nonnegativity  restriction  on  x  (t)  and  the 
production  constraint  which  is  convex  by  the  following  lemma. 

Lemma  25; 

If  f (v)  is  a  convex  increasing  function  in  v  and  $(x)  is  a 
concave  function  in  x  ,  then 


v(t)  -  F(*(x(t))) 


is  a  convex  function  in  (v,  x)  . 

Proof ; 

Since  f(’)  is  the  inverse  of  F(')  by  definition  and  f(’)  is  a 
convex  increasing  function,  F(')  is  a  concave  increasing  function. 


t 


W 
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A  concave  increasing  transformation  of  a  concave  function  is 
concave,*  and  thus  F($(x(t)))  is  concave  in  x  and 
v(t)  -  F($(x(t)))  is  convex  in  (x,v)  . 

tc  k  k  k 

Hence  the  strategy  sets  S2  are  convex  in  (2  ,  x  ,  v  )  ,  k  c  K  .  The 
additional  global  constraints  in  Example  2  on  the  factors  of  production  that 

It 

are  limited  (4.1-12)  are  convex  since  they  are  linear  in  x  .  Finally,  the 

k  k 

objective  function  is  concave  since  it  is  linear  in  x  and  v  ,  and 

k 

concave  in  2  by  Lemma  24. 

The  last  condition  to  verify  for  the  theorems  of  Chapter  3  is  that  the 
maxima  of  the  Lagrangian  subproblems  is  taken  on  for  all  u  >,  0  .  However, 
as  this  depends  on  the  solution  technique  employed,  we  will  consider  this 
point  in  the  next  section.  First,  we  will  show  the  subproblems  for  Example  1 
and  Example  2  can  be  considered  to  have  the  same  structure.  The  Lagrangian 
subproblems  for  Example  1  are 


(4.2-4) 


Max  J  !rk  vk(t)  -  Pk(p)  fk(vk(t))  - 


-  <ck  +  UB>  2k(t)  [s<2k(t)  +  Yk(l-S(2k(t))) 


l]|  e'ot  dt 


subject  to 


_k  /  k  k  1  .  k  k ,  x  ,.k 
S.  -  \z  ,  v  L  <  z  (t)  <M 


t  e  [0,  T] 


0  <,  vk(t)  £  Min  ( yk (0)  +  J  2k(s)ds,  cNt) 


where  uD  is  the  current  Lagrange  multiplier  associated  with  the  global 

D 


See  Berge  [6],  Page  191. 
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budget  constraint.  The  Lagrangian  subproblems  for  Example  2  are 

(4.2-5)  Max  ^  |rk  vk(t)  -  pk  •  xk(t)  - 

-  (c  +  ufi)  zk(t)  |^5 (zk(t) )  +  Yk(l-6(rk(t)))j|  e"0t  dt 


and  u  e  EJ  is  the  vector  of  Lagrange  multipliers  associated  with  the 
constraints  on  the  *  outs.  If  we  have  formulated  the  problem  correctly,  the 
optimal  solution  w.  .  .  be  finite;  and  thus  the  optimal  Lagrange  multipliers 
will  be  such  that  the  associated  p  ^  0  .  Although  x  (t)  is  restricted 
in  Example  2,  it  is  only  restricted  to  nonnegativity  in  the  associated 
Lagrangian  subproblems.  Hence  we  may  use  the  cost  function  representation 
to  reduce  the  Lagrangian  subproblems  ror  Example  2  to 


(4.2-6) 


Max 


|  [rk  vk(t)  -  Pk(p)  fk(vk(t))  - 

-  (c  +  uB)  zk(t)  [<5(zk(t))  +  yk(l-6(zk(t)))]j 


-at 

e 


dt 


64 


subject  to 
/  . 


k.  W  k 

i  1  *  (O  <,  mk 


t  c  [0,  T] 


0  <  vk(t)  <  Min  |yk(0)  +  ^  zk(s)ds,  £(t)j 


'  |(  1/  *' 
where  ■=  p^  + 


i  e  I  . 


Note  that  (4.2-6)  and  (4.2-4)  have  the  same  form. 
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4.3  SOLUTION  OF  THE  SUBPROBLEMS 

Ir,  order  to  emphasize  the  alternatives  presented,  we  will  note  two 
solution  techniques  for  the  Lagrangian  subproblems  -  one  exact  and  one 
approximate.  For  the  exact  technique,  the  strategy  set  will  be  restricted 
to  strategies  of  expansion  only,  i.e., 

(4.3-1)  0  -  Lk  <  zk(t)  <  Mk  V  t  c  (0,  T]  k  e  K 

and  the  homothetic  production  structure  will  be  restricted  to  constant 
returns  to  scale, *  i.e., 


(4.3-2) 


f  (v )  *  v  . 


'  k  k  -  '  k 

Noting  that  *=  +  a^  u^  for  Example  2  and  letting  p^  =  p^  for 

Example  1,  the  Lagrangian  subproblems  for  both  examples  may  then  be  written 


(4.3-3) 


-  PK(pk))  vk(t)  -  (c  +  uB)  zk(t)  | 


-at 

e 


dt 


subject  to 


0  <  zk(t)  <  Mk 


t  c  [0,  T] 


0  <  vk(r)  <  Min |yk(0)  +  f  zk(s),  £k(t)j 


k  k  'k  th 

if  r  <  p  (p  )  ,  then  the  optimal  solution  to  the  k  subproblem  is 

zk(t)  *  0  V  t  e  [0,  T]  .  If  rk  >  Pk(pk)  ,  then  the  optimal  solution  to 

the  k^  subproblem  will  be  such  that 


t 


See  Lemma  D  of  Appendix  B. 


66 


(4.3-4)  vk(t)  -  Min[yk(0)  +  f  zk(s),  Ck(t)l  t  £  [0,  T] 


)j  t  £  [0, 


and  (4.3-3)  reduces  to 


(4.3-5)  Max  /  !(rk  -  pk(p 


(pk))Min^yk(0)  +  J  zk(s)ds,  £k(t)j  - 


,  k  .  *  .  k.  .  -at  . 

-  (c  +  ufi)  z  (t)  e  dt 


subject  to 


Sk  -  Uk  |  0  <  zk(t)  <  Mk,  t  e  [0,  T  J ) 


An  exact  solution  technique  for  precisely  this  problem  (4.3-5)  has  been 
given  by  Arrow,  Beckmann,  and  Karlin  (1].  Their  method  employs  a  mlnlmax 
theorem  and  considers  the  following  problem,  which  is  equivalent  to  (4.3-5). 


Max  Min  J  } 
k  „k  k  .k  0  ( 


(1  -  a(t))(rk  -  Pk(pk)K(t) 


z  cS  a  cA 


♦  a(t)(r 


k  -  rk(pk))^yk(0)  ♦  |  zk(s)dsj 


(4.3-6) 


-(ck  +  ug)  zk(t)  e  at  dt 


where 


Sk  -  {zk  |  0  <  zk(t)  <  Mk,  t  c  [0,  T) ) 

Ak  -  (ak  |  0  <  ak(t)  <  1  ,  t  c  [0,  T] )  . 
k  k 

The  strategy  sets  S  and  A  are  compact  in  the  weak*  topology  by 

t  k  k  k  k  k 

Alaoglu's  Theorem,  and  since  the  functional  H  (z  ,  a  )  defined  on  S  *  A 

is  linear  in  each  variable  separately,  it  is  continuous  in  each  in  the  weak* 


^See  Berge  16]  p.  262 
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topology.  Hence  the  indicated  Max  in  (4.3-5)  exists.  Therefore,  the 

problem  of  optimal  capacity  expansion  of  a  multi-commodity  firm  with 

constant  returns  to  scale  can  then  be  handled  by  an  algorithm  proposed  in 

this  paper  with  the  technique  of  Arrow,  Beckmann,  and  Karlin  employed  to 

solve  the  subproblems  at  each  iteration.  In  Example  2,  the  solution  of 

the  Lagrangian  subproblems  must  be  translated  into  the  required  form  if  a 

primal  algorithm  is  to  be  employed.  This  is  easily  accomplished,  since  a 

k 

vector  of  inputs  x  (t)  yielding  minimum  cost  at  time  t  is  given  by 

(4.3-7)  xk(t)  -  fk(vk(t))  -  t  c  [0,  T] 

f  (1) 

“•]c  i 

where  x  is  determined  by 

pk  xk  *=  Min  pk,x  k  c  K  . 

(4.3-8)  Subject  to 

Fk($k(x))  >  1  . 

We  will  now  consider  the  approximate  technique  of  dynamic  programming 

for  the  solution  of  the  Lagrangian  subproblems  of  the  form  (4.2-6).  The 

k 

continuous  demand  functions  5  (t)  are  then  replaced  by  discrete 
approximations,  with  intervals  normalized  to  length  one,  and  numbered  in 
the  reverse  order,  such  that 

^  5\t)dt  tn 

(4.3-9)  Ck  -  - -  -  f  «k(t)dt  k  e  K  . 

n  n+1  J 

n+1 

^See  Lemma  E  of  Appendix  B  . 
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A  typical  approximation  is  given  below  where  the  number  of  intervals  is 
chosen  by  the  degree  of  accuracy  desired. 


If  we  define  yk  as  the  capacity  for  the  k**1  commodity  in  the  ntk 
interval,  we  have 

(4.3-10) 


n 


^n+1  +  Zn+1 


where  z^  is  the  adjustment  that  takes  place  in  the  ntk  interval.  Letting 
vk  be  the  level  of  production  of  the  ktk  commodity  in  the  ntk  interval 
and  6  be  the  discount  factor  associated  with  interest  rate  a  ,  the 
approximation  to  the  ktk  Lagrangian  subproblem  is 


Max  l  |rk  vk  -  Pk(pk)  fV)  - 
n=l  n  n 


-  (ck  +  u_)  zk(6(zS  +  Yk(l-S(zk)))!  6° 
d  n  n  n  ) 


(4.3-11)  subject  to 

k 


(  k  k 

.  k  k  k  ) 

1  z  ,  V 

L  <  z  <  M  / 

) 

=  n  ®  1 

„  k  /  k  k\( 

/ 

0  <  v  <  Min [ y  ,  £  )| 

\ 

■=  n  =  yn  n  // 

where  p.  =  p.  +  a,  u. 

ri  ri  i  x 


i  c  I 


Since 


This  is  a  straight  forward  one  dimensional  problem  that  is  relatively  easy 
to  solve  on  a  computer.  Hence,  both  examples  may  be  handled  by  the  algor Ithr 
proposed,  with  dynamic  programming  used  to  approximate  the  solutions  of  the 


Lagrangian  subproblems. 
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APPENDIX  A 
MINIMAX  THEOREMS 


We  state  here  the  two  powerful  minimax  theorems  employed  in  the 
derivation  of  the  duality  theory;  one  due  to  Sion[26)  and  the  other  to  Fan 
114]. 

Lemma  A:  Sion 's  Minimax  Theorem 

Let  M  and  N  be  convex  topological  spaces,  one  of  which  is  compact, 
and  F(u,v)  a  functional  defined  on  M  x  N  ,  quasi-concave  and  upper 


semi-continuous 

in  u 

for 

all 

veN,  quasi-convex  and  lower 

semi- continuous 

in  v 

for 

all 

ueM.  Then 

Sup  Inf  F(u,v)  «=  Inf  Sup  F(u,v)  . 

ueM  veN  veN  ucM 

Lemma  B:  Fan's  Minimax  Theorem 

Let  M  be  a  convex  set  ?  id  N  a  convex  compact  Hausdorf  space.  Let 
F(u,v)  be  a  functional  defined  on  M  x  N  ,  concave  in  u  for  all 
veN,  convex  and  lower  semi-continuous  in  v  for  all  u  e  M  .  Then 

Sup  Inf  F(u,v)  =  Inf  Sup  F(u,v)  . 

ueM  veN  veN  ueM 
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APPENDIX  B 

HOMOTHETIC  COST  AND  PRODUCTION  FUNCTIONS 

Homothetic  cost  and  production  functions  wore  first  defined  by  Shephard 
[ 2 A ]  and  are  extensively  treated  in  his  recent  work  [25].  In  this  appendix, 
we  define  the  functions,  give  some  examples,  and  note  the  properties  that 
are  employed  elsewhere. 

Definition: * 

A  homothetic  production  function  is  one  of  the  form  F(#(x))  where 
4> (x)  is  nonnegative,  homogeneous  of  degree  one,  nondecreasing  in  x  , 
upper  semi-continuous  and  quasi-concave  for  all  x  t  D  =  { x | x^  >  0  Vi); 
and  F(’)  is  nonnegative,  continuous  and  strictly  increasing  with 
F(0)  -  0  . 

In  what  follows  the  inverse  function  of  F(‘)  ,  which  always  exists  since 
F(-)  is  strictly  increasing,  will  be  denoted  by  f(-)  .  The  following 
lemma  gives  the  most  important  property  of  homothetic  production  functions. 

Lemma  C: 

The  isoquant  for  any  output  rate  u  >  0  of  a  homothetic  production 
function  may  be  obtained  from  that  for  unit  output  rate  by  radical 
expansion  from  the  origin  in  a  fixed  ratio  f(u)/f(l)  . 

The  class  of  homothetic  production  functions  includes  all  of  the  production 
functions  commonly  employed.  Some  examples  are  as  follows: 

^All  definitions  and  lemmas  are  taken  from  Shephard  [25] 
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Cobb-Douglas 


n  a 


B-l 


4>(x) 

f(u) 


A  n  x  A  >  0,  a  >  0,  [  a. 

i-1  1  i-1  1 


1/r 


Arrow- Chenery-Minhas-Solow  (ACMS) 


B-2 


*(x) 
f(u)  -  u 


l  ai  x< 

i-1 


-b 


1 

b 


a^  >  0  b  >  -  1 


Uzawa  (Generalized  ACMS) 


4>(x) 


B-3 


J  r 

-  \i 

J-l  [icNj 


ai  Xi 


.t  >  0.  bj  >  -  1.  (J  >  0,  £  ^ 
f(u)  -  U 


p.  *>  1 


The  function  f(‘)  completely  discribes  the  returns  to  scale  as  follows. 

Lemma  D: 

f (u)  convex  implies  nonincreasing  returns  to  scale,  f(u)  concave 
implies  nondecreasing  returns  to  scale,  and  f(u)  linear  implies 
constant  returns  to  scale. 

For  any  price  direction  p  e  D  =  { p | p ^  >0  Vi}  and  nonnogative  level 
of  output  u  >  0  ,  there  is  associated  with  any  production  function  a  cost 
function  given  by 
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Q(u,  p)  ■  Min  p*x 

B-4  F(4>(x))  >  u 

x  e  D 

Note  that  the  cost  function  is  defined  for  x  >  0  but  otherwise  unrestricted. 
For  a  horaothetic  production  function  the  cost  function  has  a  more  specific 
form  given  by 


B-5  Q(u,  p)  <=  P(p)  f(u)  . 

This  convenient  representation  for  the  cost  function  and  the  radial  expansion 
property  of  Lemma  G  yield  the  following  useful  result. 


Lemma  E: 


If  the  vector  x(l)produccs  one  unit  of  output  at  minimum  cost  then  the 
vector  x(u)  =  x(1)  produces  u  units  of  output  at  minimum  cost. 


Proof : 


By  definition  Q(u,  p)  =  Min  p-x  =  p-x(u)  and 

F(<Kx))>u 

function  Q(u,  p)  *  P(p)  f(u)  .  Therefore, 


P(p) 


*±vl 

f  (u) 


p 


fa) 


for  a  homothetic  cost 


and  hence 

x(i) 
f(i)  xu' 


p  •  x(u)  c  p  ’  ’fQj"  x(l)>  Therefore,  the  input  vector 
produces  u  units  of  output  at  minimum  cost. 


Corresponding  to  each  of  the  examples  of  homothetic  production  functions,  w<j 
have  the  following  homothetic  cost  functions: 
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Cobb- Douglas 


B-6 


Q(u,  p)  - 

A  >  0 


ul/r 


A rrow- Chenery-Minhas- Solow  (A CMS) 


B-7 


Q(u,  p) 


>  0 


Uzawa  (Generalized  ACMS) 


B-8 


ai  >  °*  bj 


1,  P,  >  0,  l  p.  -  1  . 

J  j-1  J 
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